Some Tools Curated on GitHub
Category Programming Technology
Technical Sites
- Hacker News: An excellent link aggregation site for programming.
- Programming reddit: Same as above.
- MSDN: Microsoft's official technical hub, mainly document-oriented.
- infoq: Enterprise-level applications, focusing on software development.
- OSChina: Open-source technology community, doing well in open-source aspects.
- cnblogs, 51cto, csdn: Common technical communities, each with its specialties.
- stackoverflow: IT technical Q&A website.
- GitHub: The world's largest source code management platform, hosting many famous open-source projects like the Linux kernel, OpenStack, etc.
- Free IT eBooks: http://it-ebooks.info/
- DevStore: Developer service store.
Recommended Books
- Peopleware
- The Mythical Man-Month
- Code Complete 2
- The Art of Computer Programming
- The Pragmatic Programmer
- The Developer's Code
- Effective Programming: More Than Writing Code
- Computer Systems: A Programmer's Perspective
- Joel on Software
- Introduction to Algorithms (MIT Press)
- Discrete Mathematics and Its Applications
- Design Patterns
- Beautiful Code
- Hackers and Painters
- Programming Pearls
- C++ Primer
- Effective C++
- TCP/IP Illustrated
- The Art of Unix Programming
- Introductory Lectures on Psychoanalysis by Freud
- Getting Things Done: The Art of Stress-Free Productivity
Platform Tools (All open-source)
- Redmine/Trac: Project management platforms.
- Jenkins/Jira (non-open-source): Continuous integration systems (Apache Continuum, part of Apache, not yet studied).
- Sonar: Code quality management platform.
- git, svn: Source code version control systems.
- GitLib/Gitorious: Build your own GitHub server.
- gitbook: https://www.gitbook.io/ Great for writing books, also good for documentation.
- Travis-ci: Essential for open-source project continuous integration, integrated with GitHub, https://travis-ci.org/
- Open-source testing tools, communities (Selenium, OpenQA.org)
- Puppet: An automation management engine, suitable for Linux, Unix, and Windows platforms. Manages resources like files, users, processes, software packages across any number of machines.
- Nagios: System status monitoring and alerting, also Icinga (fully compatible with nagios, simpler configuration, powerful).
- Ganglia: Distributed monitoring system.
- fleet: Distributed init system.
Web Crawling Tools (Fun tools)
- Phantomjs
- berserkJS (improved version based on Phantomjs)
- SlimerJS
- CasperJS
- selenium
Web Server Performance/Stress Testing Tools/Load Balancers
- http_load: Very small program, less than 100K after decompression.
- webbench: A website stress testing tool under Linux, can simulate up to 30,000 concurrent connections.
- ab: A powerful testing tool included with apache.
- Siege: An open-source stress testing tool, can configure concurrent user access to a web site, records response times, and repeats under a certain number of concurrent accesses.
- squid (front-end cache), nginx (load balancer), nodejs (can also be used, write some code to achieve high-performance load balancing): Common load balancers.
- Piwik: Open-source website traffic statistics system.
- ClickHeat: Open-source website click heatmap.
- HAProxy: High-performance TCP/HTTP load balancer.
- ElasticSearch: Search engine based on Lucene.
- Page Speed SDK and YSLOW
- HAR Viewer: HAR analysis tool.
- protractor: E2E (end to end) automated testing tool.
Web Front-end Related
- GRUNT: js task runner
- Sea.js: js modularization
- knockout.js: MVVM development for the front end, binding technology
- Angular.js: Developing WEB applications with dynamic HTML & JS!
- Highcharts.js, Flot: Common Web chart plugins
- Raw: A very good advanced data visualization tool
- Rickshaw: Time series chart library, can be used to build real-time charts
- JavaScript InfoVis Toolkit: Another Web data visualization plugin
- Pdf.js, display pdf in html
- ACE, CodeMirror: Html code editor (ACE is very good)
- NProcess: Gorgeous loading progress bar
- impress.js: Create stunning content display effects (similar to reveal)
- Threejs: 3D Web library
- Hightopo: 2D, 3D visualization UI library based on Html5
- jQuery.dataTables.js: Highly flexible table plugin
- Raphaël: js, canvas drawing library, later found that Baidu Index graphics are drawn with it
- director.js: js routing module, front-end routing, Nodejs backend routing, etc., suitable for constructing single-page applications
- pace.js: Page loading progress bar
- bower: Web package manager
- jsnice: Interesting js decompilation tool, guessing compressed variable names http://www.jsnice.org/
- D3.js: A JavaScript data display library (similar to P5.js)
- Zepto.js: A jQuery alternative for mobile, or use jquery-mobile. UI frameworks: Foundation, Boostrap, Pure, EasyUI, Polymer Front-end UI designer must-go websites: Dribbble, awwwards, unmatchedstyle, UIMaker Mozilla Developer Center: https://developer.mozilla.org/en-US/ Icon resources: IcoMoon (my favorite), Themify Icons, FreePik, Glyphiconsart
- Dialog: Very beautiful dialog box
- AdminLTE: An open-source project on github, based on Boostrap3's backend management page framework
- Respond.js: Make IE6-8 support responsive design
- require.js: js module loading library
- select2: A selection box alternative library with more features than chosen
- AngularUI: UI library integrated with angular.js
- normalize.css: A library that adopts modern standards to make browser rendering of html consistent
- CreateJS: Html5 game engine Less, Compass: Simplify CSS development
- emojify.js: Automatically recognizes Emoji text on the web and displays it as an image
- simditor: A nice open-source html editor, simple and efficient
- Sencha: Mobile development framework based on html5
- SuperScrollorama+TweenMax+skrollr: Create super cool parallax scrolling effect web animation
- jquery-smooth-scroll: Same as above, smooth scrolling plugin
- Animate.css: A css library that implements various animation effects
- Emmet: Essential for front-end engineers, the predecessor of ZenCode
- MagicDraw: Uml diagram tool
Big Data Processing/Data Analysis/Distributed Tools
- Hadoop: Distributed file system, combined with its MapReduce programming model can be used for batch processing of massive data (Hive, Pig, HBase, etc.), it's worth mentioning Cloudera's Hadoop branch CDH5, integrates Spark based on YARN MRv2, directly usable in production environments, very useful for enterprises to quickly build data warehouses.
- Ceph: Linux distributed file system (feature: no center)
- Storm: Real-time stream data processing, you can see an introduction from IBM (there's also Yahoo's S4, also for stream data processing)
- Spark: Large-scale streaming data processing (can cope with three common data processing scenarios in enterprises: complex batch data processing; interactive query based on historical data; real-time data stream processing), there's a good article on CSND
- Spark Streaming: Real-time computing framework based on Spark
- Tachyon: Distributed memory file system
- Mesos: A cluster manager for computing frameworks, provides effective resource isolation and sharing across distributed applications or frameworks. Impala: A new generation of open-source big data analysis engine, provides Sql semantics, stronger than Hive in speed.
- SNAPPY: Fast data compression system, suitable for the Hadoop ecosystem
- Kafka: High-throughput distributed message queue system
- ActiveMQ: The most popular, powerful open-source message bus made by Apache
- MQTT: Message Queuing Telemetry Transport, an instant messaging protocol developed by IBM, may become an important part of the Internet of Things
- RabbitMQ: I remember OpenStack uses this, right?
- ZeroMQ: Claims to make distributed computing simpler, a distributed message queue, you can see an introduction from Cloud Wind Open-source log collection systems: scribe, chukwa, kafka, flume. There's a comparison article
- Zookeeper: A reliable open-source project for distributed coordination
- Databus: LinkedIn's real-time low-latency data capture system Data source acquisition: Flume, Google Refine, Needlebase, ScraperWiki, BloomReach Serialization technologies: JSON, BSON, Thrift, Avro, Google Protocol Buffers NoSql: Apache Hadoop, Apache Casandra, MongoDB, Apache CouchDB, Redis, BigTable, HBase, Hypertable, Voldemort, Neo4j MapReduce related: Hive, Pig, Cascading, Cascalog, mrjob, Caffeine, S4, MapR, Acunu, Flume, Kafka, Azkaban, Oozie, Greenplum Data processing: R, Yahoo! Pipes, Mechanical Turk, Solr/ Lucene, ElasticSearch, Datameer, Bigsheets, Tinkerpop NLP natural language processing: Natural Language Toolkit, Apache OpenNLP, Boilerpipe, OpenCalais Machine learning: WEKA, Mahout, scikits.learn, SkyTree Visualization technologies: GraphViz, Processing, Protovis, Google Fusion Tables, Tableau, Highcharts, EChats (Baidu's is pretty good), Raphaël.js
- Kettle: Open-source ETL tool
- Pentaho: Open-source BI system centered on workflow
- Mondrian: Open-source Rolap server
- Oozie: Open-source Hadoop workflow scheduling engine Open-source data analysis visualization tools: Weka, Orange, KNIME Cobar: Alibaba's MySql distributed middleware
C & C++
Thrift: Used for the development of scalable and cross-language services (similar to Avro, Google protobuf). libevent: A network library triggered by events, suitable for various platforms such as windows, linux, bsd, etc., manages the event mechanism using system calls like select, epoll, kqueue, etc. (there's also libev). Boost: Not much to say, the quasi-C++ standard library
Ptmalloc\Valgrind\Purify
NetworkServer Architecture: acceptor->dispatcher->worker (this is not a tool)
breakpad: Crash dump and analysis module, widely used in many crash reports
UI-related: MFC, BCG, and QT are not mentioned, but more advanced technologies include Html and DirectUI: libcef (based on the Chrome kernel, thinking about developing pages with HTML5 is quite exciting), HtmlLayout, Duilib, Bolt, and non-C++ options like node-webkit, which integrates node and webkit kernels.
Game Development Related
MINA: Java-based mobile and web game server development (also Netty, which is also powerful, both based on NIO)
HP-Socket: Seen in some web game servers built with this
Yunfeng's tech blog: http://blog.codingnow.com/
OGRE: Famous 3D graphics rendering engine
OpenVDB: DreamWorks' C++ special effects library, open-source
cocos2d: Cross-platform 2D game engine
unity3d: Cross-platform 3D game engine, very popular
Nodejs: Also used to develop mobile and web game servers (NetEase's Pomelo is an example)
Log Aggregation, Distributed Log Collection
Scribe: Facebook's (nodejs + scribe + inotify for log synchronization)
logstash: Powerful log collection system, capable of developing a powerful log analysis platform based on logstash+kibana+elasticsearch+redis
log.io: Real-time log collection system developed with nodejs
RTP, Real-time Transport Protocol and Audio-Video
RTP, RTCP, RTSP -> librtp, JRTPLIB (follows RFC1889 standard)
Circular Buffer, used for real-time data transmission
SDL, ffmpeg, live555, Speex
Red5: Open-source Flash streaming media server developed in Java. It supports: converting audio (MP3) and video (FLV) into playback streams; recording client playback streams (FLV only); shared objects; live streaming distribution; remote calls.
Python
Eric, Eclipse+pydev, good Python IDEs
PyWin: Win32 api programming package
numpy: Scientific computing package, mainly for large matrix calculations, also SciPy, Matplotlib
GUI-related: PyQt, PyQwt
supervisor: Process monitoring tool
Java-related
Common IDEs: IntelliJ IDEA, Eclipse, Netbeans
Web development-related: Tomcat, Resin, Jetty, WebLogic, common components Struts, Spring
HibernateNetty: Asynchronous event-driven network application programming framework, better for high-concurrency network programming (NIO framework)
MINA: Simplifies the development of high-performance and reliable network applications (also an NIO framework), many mobile game servers are developed with it
jOOQ: java Orm framework
Activiti: Workflow engine, similar to jBPM, Snaker
Perfuse: A user interface package for displaying structured and unstructured data in an interactive graphical format.
Gephi: Complex network analysis software, mainly used for various networks and complex systems, interactive visualization and exploration of dynamic and hierarchical graphs
Nutch: Famous crawler project, hadoop developed from this project
web-harvest: Web data extraction tool
POM tool: Maven+ArtifactoryNetflix
Curator: Netflix's open-source Zookeeper client library, simplifying Zookeeper client programming
Akka: A concurrency framework based on the actor model
EclEmma: Coverage testing tool
.net-related
Xilium.CefGlue: .NET wrapper based on the CEF framework, developing Chrome kernel browsers based on .NET
CefSharp: Same as above, has a WebKit wrapper, C# and Js interaction becomes simpler
netz: Free .NET executable compression tool
SmartAssembly: Extremely powerful .net code optimization and obfuscation tool
NETDeob0: .net de-obfuscation tool, truly the devil is one foot high and the Taoist priest is ten feet high (also de4dot, on GitHub, both open-source)
ILMerge: Combines all referenced DLLs and exe files into one exe file
ILSpy: Open-source .net program decompiler
Javascript.NET: Excellent js execution engine, encapsulates v8
NPOI: Excel operation
DotRAS: Remote access service module
WinHtmlEditor: Html editor for Winform
SmartThreadPool: Thread pool with advanced features implemented in C#
Snoop: WPF Spy Utility
Autofac: Lightweight IoC framework
HtmlAgilityPack: Powerful Html parser
Quartz.NET: Job scheduling
HttpLib: @CodePlex, simplifies http requests
SuperSocket: Simplifies Socket operations, also has SuperWebSocket for developing standalone WebSocket servers
DocX: Manipulate Word files without installing Office
Dapper: Lightweight ORM class, good performance
HubbleDotNet: Full-text search system supporting database access
fastJSON: @CodeProject, high-performance json serialization class
ZXing.NET: @CodePlex, QR, barcode related
Nancy: Lightweight Http server, can get rid of IIS for small Web applications (Nancy.Viewengines.Razor, can add Razor engine)
AntiXSS: Microsoft's XSS defense library
Microsoft Web Protection Library
Jint: JavaScript interpreter
CS-Script: Execute C# code files as scripts
Jexus: High-performance, easy-to-use, free ASP.NET server on Linux
Clay: Makes dynamic even more flexible, writing C# like js
DynamicJSON: Get json data without defining data models
Antlr: Open-source parser generator (not suitable for C# only, other languages can use it too)
SharpPcap: C# version of WinPcap, powerful network packet analysis library (comes with PacketNotNet for packet protocol analysis)
Roslyn: C#, VB compiler
ImageResizer: Freely control image size on the server side, truly a神器, convenient for mobile to send small images, PC to send large images, CMS uses it very conveniently
UI-related: DevExpress, Fluent (Office 07 style), mui (Modern UI for WPF)
NetSparkle: Application auto-update component
ConfuserEx: Open-source .net obfuscation tool
ServiceStack: Open-source high-performance Web service framework, can be used to build high-performance REST services
Expression Evaluator: Eval for C#, processes string expressions
Common Tools
Fiddler: Excellent Web front-end debugging tool, of course for the underlying http protocol, usually Chrome's built-in debugging tools are sufficient, special cases need to be handled with it
wireshark: Famous network packet analysis tool
PowerCmd: A great replacement for Windows Cmd
RegexBuddy: Powerful regular expression testing tool
Soure Insight: Source code reading神器
SublimeText: Programmers' favorite editor
Database.NET: A universal relational database client, developed based on .NET 4.0, simple processing is quite convenient
Navicat Premium: Client supporting MySql, PostgreSQL, Oracle, Sqlite, and SQL Server, less versatile than Database.NET, but performance is much better than Database.NET, with built-in backup function for timed database backup
Synergy: Controls multiple computers with one set of keyboard and mouse within a LAN
DameWare: Remote assistance toolset (mainly used to control large screens at my company)
Radmin: Remote control tool, used it for a while
DameWare, had to crack it, not well supported on Win7, found this one better
Listary: A "super神器" that greatly improves your Windows file browsing and search speed efficiency
Clover: Adds tabs to the file explorer
WinLaunch: Simulates Mac OS's Launch tool
Fritzing: Circuit diagram drawing tool
LICEcap: gif tutorial maker
git, svn: Version control systems
Enigma Virtual Box (encapsulates exe, dll, etc. into one executable program)
Open DBDiff (for SqlServer) database synchronization
SymmetricDS: Database synchronization
BIEE, Infomatica, SPSS, weka, R language: Data analysis
CodeSmith, LightSwitch: Code generation
Pandoc: Markdown conversion tool, used for publishing books. Used to play with docbook, but now Markdown is popular
Window Magnet [Mac]: Enhances Mac window management, like Win7 with the ability to drag windows to screen edges to auto-adjust
log explorer: View SqlServer logs
dependency walker: Query Windows application dll dependencies
Shairport4w: Transmits audio from iPhone, iPad, iPod to PC via AirPlay protocol
ngrok: Intranet penetration tool
Axure: Rapid prototyping tool, also an online drawing tool from a domestic startup team, used it, very good http://www.processon.com
tinyproxy: (Linux) Small proxy server supporting http and https protocols
EaseUS Partition Master: Super simple partition adjustment tool, fast speed, not enough C drive space, use it to get some space from the D drive, no need to reinstall the system
Cheat Engine: Essential tool for modifying memory values while playing games (used it when playing Xuanyuan Sword 6, very convenient)
ApkIDE: Android decompilation神器
VPN tools (自由门, 天行浏览器)
Design tools: Sketch, OmniGraffle
MindManger: Mind mapping
** Click me to share notes
-
-
-