# jeroenjanssens/data-science-at-the-command-line

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/jeroenjanssens-data-science-at-the-command-line).**

3,952 stars · 767 forks · HTML · NOASSERTION

## Links

- GitHub: https://github.com/jeroenjanssens/data-science-at-the-command-line
- Homepage: https://datascienceatthecommandline.com
- awesome-repositories: https://awesome-repositories.com/repository/jeroenjanssens-data-science-at-the-command-line.md

## Topics

`bash` `book` `bookdown` `cli` `cowsay` `curl` `data-science` `ggplot2` `gnuplot` `jq` `linux` `oreilly` `oreilly-books` `python` `r` `shell` `terminal` `unix` `zsh`

## Description

This project provides a framework for performing data science tasks using command-line tools and scripts. It focuses on the processing and analysis of text and structured data directly within the terminal.

The approach centers on using Unix pipes to stream data between independent processes and employing shell scripting to automate repetitive data science workflows. It utilizes plain-text interchange formats, such as CSV, to move information between diverse utilities.

Capability areas include text-based data processing, command-line data analysis, and terminal-based data visualization. These are achieved by chaining discrete executable programs into linear transformation pipelines.

## Tags

### Development Tools & Productivity

- [Command Piping](https://awesome-repositories.com/f/development-tools-productivity/command-piping.md) — Connects independent command-line utilities by streaming data from one process to another using Unix pipes.
- [Scripted Workflow Automations](https://awesome-repositories.com/f/development-tools-productivity/task-automation-tools/scripted-workflow-automations.md) — Automates repetitive data science tasks by chaining terminal commands into repeatable executable scripts.
- [Standard Stream Interfaces](https://awesome-repositories.com/f/development-tools-productivity/standard-stream-interfaces.md) — Leverages standard input and output streams to decouple data producers from consumers within the pipeline.
- [Text Processing Utilities](https://awesome-repositories.com/f/development-tools-productivity/terminal-shell-cli/terminal-cli-enhancements/shell-terminal-utilities/general-enhancements-tooling/terminal-utilities/text-output-processing/text-processing-utilities.md) — Cleans and transforms raw text files into structured formats using standard command-line utilities.

### Part of an Awesome List

- [Data Analysis and Processing](https://awesome-repositories.com/f/awesome-lists/data/data-analysis-and-processing.md) — Performs data manipulation and visualization tasks directly within the terminal using scripts and utilities. ([source](https://datascienceatthecommandline.com/))
- [Data Science and Analysis](https://awesome-repositories.com/f/awesome-lists/data/data-science-and-analysis.md) — Provides a framework for data exploration, manipulation, and analysis using command-line tools. ([source](https://cdn.jsdelivr.net/gh/jeroenjanssens/data-science-at-the-command-line@master/README.md))
- [Data Wrangling and Processing](https://awesome-repositories.com/f/awesome-lists/data/data-wrangling-and-processing.md) — Brings data science workflows to the terminal environment.

### Data & Databases

- [Command-Line Data Processors](https://awesome-repositories.com/f/data-databases/command-line-data-processors.md) — Analyzes datasets using high-performance terminal tools for quick calculations and data manipulations.
- [Plain-Text Data Handlers](https://awesome-repositories.com/f/data-databases/plain-text-data-handlers.md) — Uses universal plain-text formats like CSV and TSV for seamless data exchange between diverse terminal tools.

### Operating Systems & Systems Programming

- [Standard Stream Piping](https://awesome-repositories.com/f/operating-systems-systems-programming/standard-stream-piping.md) — Creates linear transformation pipelines by streaming data between independent processes using standard I/O.
- [Shells & Scripting](https://awesome-repositories.com/f/operating-systems-systems-programming/terminal-command-line-environments/shells-scripting.md) — Uses shell scripting to orchestrate sequences of command-line calls for data cleaning and processing.

### Software Engineering & Architecture

- [Modular Program Composition](https://awesome-repositories.com/f/software-engineering-architecture/modular-program-composition.md) — Implements a modular architecture by chaining independent command-line utilities into complex analysis workflows.
- [Text Stream Manipulations](https://awesome-repositories.com/f/software-engineering-architecture/coding-patterns/pattern-based-code-manipulation/text-stream-manipulations.md) — Processes data as sequences of characters and lines for rapid filtering and transformation via shell pipes.

### User Interface & Experience

- [Terminal Data Visualizations](https://awesome-repositories.com/f/user-interface-experience/terminal-data-visualizations.md) — Renders simple charts and visual representations of data directly within the command line interface.