How I Built My Own Shell in Java

Back in November, I picked up the CodeCrafters "Build Your Own Shell" challenge because I wanted to stop treating the terminal like magic.

I use the terminal all the time. I run ls, cd, pwd, grep, cat, and dozens of other commands every week. But for a long time, the experience was mostly intuitive, not deeply understood. I knew what to type, but I did not fully understand what a shell actually does after I press Enter.

So I built one.

This project is my attempt to recreate the core behavior of a Unix-like shell in Java. It is not a full replacement for Bash or Zsh, and it is not a terminal emulator in the strict sense. That distinction matters:

• A terminal emulator is the program that gives me a text window and handles keyboard input, ANSI colors, cursor movement, and screen rendering.
• A shell is the command interpreter that runs inside that terminal and decides what to do with text like echo hello, cd src, or cat file.txt | grep foo.

What I built here is the second piece: the shell.

And that turned out to be one of the most educational systems projects I have worked on.

What This Project Can Do Right Now

As of the current version of the codebase, my shell supports:

• A REPL loop with a shell-style prompt
• Built-in commands implemented directly in Java
• External program execution through the operating system
• Command lookup through PATH
• Quoted strings and backslash escaping
• Pipelines using |
• Standard output redirection using > and 1>
• Standard output append redirection using >> and 1>>
• Standard error redirection using 2>
• Standard error append redirection using 2>>
• A cross-platform-ish installation flow for Unix-like systems and Windows
• A fairly solid JUnit test suite around the current implementation

The built-in commands currently implemented are:

• exit
• echo
• type
• pwd
• cd

The High-Level Mental Model of a Shell

Before I get into the Java code, it helps to define what a shell actually does.

At a high level, a shell repeatedly performs the same cycle:

1. Read a line of input from the user.
2. Break that line into meaningful tokens.
3. Detect operators like pipes and redirection.
4. Decide whether the command is built into the shell or should be delegated to the OS.
5. Execute the command.
6. Print the result and wait for the next line.

That is the classic REPL model:

• Read
• Eval
• Print
• Loop

In my project, that entire flow lives inside the Shell class, while the parser, built-ins, process execution, and pipeline logic are split into separate packages.

Project Structure

The codebase is organized into a few focused packages:

src/main/java/
├── Main.java
└── com/adi/shell/
    ├── core/
    │   ├── Shell.java
    │   └── ShellContext.java
    ├── parser/
    │   └── CommandParser.java
    ├── command/
    │   ├── Command.java
    │   ├── CommandRegistry.java
    │   ├── ExternalCommandExecutor.java
    │   └── builtin/
    │       ├── ExitCommand.java
    │       ├── EchoCommand.java
    │       ├── TypeCommand.java
    │       ├── PwdCommand.java
    │       └── CdCommand.java
    ├── pipeline/
    │   └── PipelineExecutor.java
    └── io/
        └── StreamUtils.java

I like this structure because it maps almost perfectly to the shell lifecycle:

• core manages the shell loop and shell state.
• parser converts raw user input into tokens.
• command defines what a command is and how built-ins are registered.
• builtin contains commands that must run inside the shell process.
• pipeline handles cmd1 | cmd2.
• io contains reusable stream helpers.

The Entry Point: Booting the Shell

The entry point is intentionally tiny:

public static void main(String[] args) throws Exception {
    new Shell().run();
}

That is a good design choice.

Main.java should not know how tokenization, piping, path lookup, or redirection work. It should only start the application. All of the real behavior is delegated to Shell.

The Core Engine: Shell.java

Shell is where the whole program gets wired together.

When the shell starts, it creates:

• a ShellContext
• a CommandRegistry
• a CommandParser
• a PipelineExecutor
• an ExternalCommandExecutor

It also registers all built-in commands:

registry.register("exit", new ExitCommand());
registry.register("echo", new EchoCommand());
registry.register("type", new TypeCommand(registry));
registry.register("pwd", new PwdCommand());
registry.register("cd", new CdCommand());

Shell State: Why cd Needs ShellContext

One of the first things I learned building this project is that not every command can be treated the same way.

For example, if I run:

cd src

the shell's own current working directory needs to change.

If I launch cd as a separate child process, that child process can change its working directory, but the parent shell will remain where it was. That would make cd useless.

That is why commands like cd, pwd, and exit need to exist as built-ins inside the shell itself.

To support that, my project uses ShellContext.

ShellContext currently stores the current working directory as a File. On startup it uses System.getProperty("user.dir"), and when cd succeeds it updates both:

• the internal currentDir
• the Java system property user.dir

That gives the rest of the shell a single place to read the current directory from.

This is a small class, but conceptually it is very important. It separates mutable shell state from the rest of the execution logic.

Defining Commands with an Interface

All commands implement a common interface:

void execute(String[] args, ShellContext context);
void executeInPipeline(List<String> args, InputStream in, OutputStream out, ShellContext context);

That second method is especially interesting.

It means a command can run in two contexts:

• normal interactive execution
• pipeline execution

This is how built-ins like echo and pwd can participate in pipelines without being forked into separate OS processes.

That design makes the shell more modular. It also mirrors a real systems-programming idea: a command is not just "something that prints to the terminal". A command is something that consumes input, produces output, and interacts with process state.

Parsing User Input: The Hardest Part of a Shell

Parsing command lines is where a shell stops being a beginner project.

At first glance, a command looks simple:

echo hello world

But real shell input quickly becomes more complicated:

echo "hello world"
echo 'hello world'
echo hello\ world
echo "a quote: \" "
echo hello > out.txt
echo hello 2>> err.txt
echo hello | cat

That means the parser has to understand:

• spaces as separators
• single quotes
• double quotes
• backslash escaping
• pipe operators
• redirection operators
• attached redirections like >file.txt

My shell does that in CommandParser.java.

How the parser works

The parser is hand-written. It scans the input string one character at a time and keeps track of state using booleans:

• inSingle
• inDouble

That lets it decide whether a character should be interpreted literally or as syntax.

For example:

• A space outside quotes ends the current token.
• A space inside quotes becomes part of the current token.
• A single quote toggles single-quote mode unless the parser is already in double quotes.
• A double quote toggles double-quote mode unless the parser is already in single quotes.

Backslash handling

Backslashes behave differently depending on context:

• Outside quotes, a backslash escapes the next character.
• Inside double quotes, the implementation only treats a few characters as escapable: ", \, $, and `.
• Inside single quotes, backslashes are preserved literally.

That is very close to real shell behavior and is one of the more subtle parts of the implementation.

Turning redirection syntax into internal markers

After tokenization, the parser performs a second pass over the token list and rewrites redirection operators into internal marker tokens:

• > or 1> becomes __REDIR__
• >> or 1>> becomes __APPEND__
• 2> becomes __REDIR_ERR__
• 2>> becomes __APPEND_ERR__

So something like:

echo hello > out.txt

gets transformed into a token stream conceptually like:

["echo", "hello", "__REDIR__", "out.txt"]

Built-In Commands

Let me walk through every built-in command and explain both the user-facing behavior and the implementation.

exit

exit ends the shell process.

Implementation-wise, it is tiny:

• exit with no argument calls System.exit(0)
• exit 42 calls System.exit(42)

In pipeline mode, it becomes a no-op.

That behavior makes sense in this implementation. If exit appeared in the middle of a pipeline and immediately terminated the shell, that would be very surprising.

One important detail: the test suite intentionally skips the tests that would actually call System.exit() because that would kill the test JVM itself.

echo

echo joins all arguments after the command name with spaces and writes the result followed by a newline.

In normal execution:

• it can print directly to standard output
• it can write to a file
• it can append to a file

EchoCommand has its own logic to peel off redirection markers from the end of the argument list before constructing the final output string.

That means echo handles redirection internally rather than delegating all of that logic to a shared redirection layer.

This works, but it is also one of the architectural tradeoffs in the codebase. Redirection is partly centralized in the parser and external executor, but partly duplicated inside EchoCommand.

In pipeline mode, echo ignores files and simply writes its result to the provided output stream.

type

type answers the question:

If I type this command name into the shell, what is it?

It checks:

1. whether the name exists in the builtin registry
2. whether an executable file with that name exists in one of the directories listed in PATH

So the outputs look like:

• echo is a shell builtin
• ls is /usr/bin/ls
• foo: not found

This command is a great example of why the registry abstraction is valuable. Because built-ins are centrally registered, type does not need to duplicate a hardcoded list of builtin names.

pwd

pwd is straightforward: it prints the absolute path of the shell's current working directory from ShellContext.

The key point is that it does not ask the operating system every time. It trusts the shell's own internal state.

That makes the shell deterministic: cd updates the context, and pwd reads the context.

cd

cd is where shell state becomes real.

The implementation supports:

• absolute paths like /home/adi/projects
• relative paths like src/main/java
• home-directory shorthand using ~

The logic is:

1. Read the target path from args[1].
2. Resolve it:
   • if it starts with /, treat it as absolute
   • if it starts with ~, resolve to HOME or user.home
   • otherwise resolve it relative to context.getCurrentDir()
3. Check whether the result exists and is a directory.
4. If valid, update the shell context to the canonical directory.
5. Otherwise, print No such file or directory.

Using getCanonicalFile() is a solid touch because it normalizes things like .. and symbolic path noise.

External Command Execution

If a command is not a registered built-in, my shell tries to run it as an external program. This happens in ExternalCommandExecutor.

The flow is:

1. Receive the parsed command tokens.
2. Strip internal redirection markers into booleans and file paths.
3. Build a clean command list.
4. Create a ProcessBuilder.
5. Set its working directory from ShellContext.
6. Configure stdout and stderr redirection if needed.
7. Start the process and wait for it to finish.

Using PATH

One subtle thing here is command lookup.

In a normal shell, typing:

ls

works because the shell searches the directories listed in the PATH environment variable.

My implementation reads PATH using System.getenv("PATH"), splits it using File.pathSeparator, and checks each directory for an executable file matching the command name.

If the initial ProcessBuilder(cmdList).start() fails with an IOException, the shell performs this manual lookup and tries again using the resolved absolute path.

If nothing is found, it prints:

command not found

Redirection for external commands

External command redirection is implemented through ProcessBuilder.Redirect.

That means:

• stdout can be inherited by the parent shell or redirected to a file
• stderr can be inherited by the parent shell or redirected to a file
• append mode is supported

The implementation also contains a fallback behavior for missing parent directories: instead of surfacing an error, it redirects output to the platform null device (/dev/null on Unix-like systems or NUL on Windows).

That is an implementation choice, and it is worth calling out because it is different from how a full shell would normally report redirection failures.

Pipelines: cmd1 | cmd2

Pipelines are where the project starts feeling like a real shell. When the shell sees |, it splits the parsed token list into multiple command stages and hands the result to PipelineExecutor.

Inside Shell.run(), the parsed token list is scanned and broken into a List<List<String>>, where each inner list is one stage of the pipeline.

For example:

echo hello | /bin/cat

becomes something conceptually like:

[
  ["echo", "hello"],
  ["/bin/cat"]
]

Supporting built-ins in pipelines

This is the most interesting part of the implementation.

If a stage is a built-in:

• the shell does not spawn a child process
• it executes the builtin directly through executeInPipeline(...)

If that builtin is not the last stage:

• its output goes into a ByteArrayOutputStream
• that byte array is then wrapped in a ByteArrayInputStream
• the next stage reads from that stream

That is a smart way to make in-process commands behave like pipe producers without requiring OS-level pipes.

Supporting external commands in pipelines

If a stage is not a built-in:

• the shell creates a ProcessBuilder
• sets the working directory
• merges stderr into stdout with redirectErrorStream(true)
• starts the process

If there is previous stage output, a feeder thread copies the previous stage's bytes into the process's stdin.

If it is the last stage, another thread copies the process's stdout to System.out.

That means the pipeline engine is manually handling stream plumbing using Java threads and streams.

This is exactly the kind of low-level systems idea I wanted to understand better when I started the project:

a pipeline is really just one command's output being connected to another command's input.

I/O Helpers: StreamUtils

StreamUtils contains two reusable helpers:

• streamCopy(...)
• writeToFile(...)

streamCopy(...) repeatedly reads from an input stream into an 8 KB buffer and writes into an output stream, optionally closing the destination when finished.

This helper is used in the pipeline engine to move bytes between commands.

writeToFile(...) is a small abstraction over BufferedWriter and FileWriter that supports overwrite and append mode.

This class is not flashy, but projects like this become much easier to reason about when repetitive stream logic is extracted into one place.

What I Learned About Real Shell Behavior

While building this project, I kept running into the fact that shells are more formal than they first appear.

According to the POSIX shell model, the shell does not just split on spaces and run a command. The rough order is closer to:

1. tokenize input
2. parse operators
3. perform expansions
4. apply redirections
5. remove redirection syntax from argument lists
6. execute the final command

My shell implements an important subset of that model:

• tokenization
• quote handling
• escape handling
• redirection recognition
• pipeline execution
• command search and execution

But it does not yet implement:

• variable expansion like $HOME
• command substitution like $(pwd)
• glob expansion like *.java
• logical operators like && and ||
• background jobs
• input redirection with <
• shell functions or aliases

That matters because it defines the current scope honestly.

This project is already a real shell implementation, but it is a subset shell, not a full Bash-compatible shell.

The Test Suite Matters More Than I Expected

One thing that stood out while analyzing the current version of the project is that the codebase now has a meaningful JUnit test suite.

I ran the current test suite locally and got:

• 83 tests run
• 0 failures
• 0 errors
• 2 skipped tests for the System.exit() behavior

That is important because shell logic is full of edge cases.

The tests cover:

• parser behavior
• redirection marker parsing
• quote handling
• escape handling
• registry behavior
• built-in commands
• shell context behavior
• external command execution
• pipeline execution
• stream utilities

The existence of this test suite also reveals something interesting about the repo history: the TODO.md file still says there are no tests, but that is no longer true. The implementation has already progressed beyond some of the older documentation.

Build and Packaging Details

The project uses Maven and currently builds a fat JAR named mini-shell.jar.

There are a few notable details in the build setup:

• The current pom.xml is configured for Java 21.
• The runtime scripts still pass --enable-preview.
• A JLine dependency is declared, but the shell currently reads input using Scanner.
• The Maven Assembly Plugin packages dependencies into a runnable JAR.

There are also helper scripts:

• install.sh for Unix-like systems
• install.bat for Windows
• Shell.sh as a local launcher
• your_program.sh for CodeCrafters-style local execution

This part of the project taught me that shipping software is a separate engineering concern from writing core logic. It is one thing to make a shell run in IDE -> Run. It is another to package it so it can be installed and executed from a terminal like a normal command.

A Few Honest Limitations in the Current Implementation

I want to be precise here, especially because this write-up is going on my portfolio.

The shell is functional and thoughtfully structured, but it also has some real limitations in its current state:

• Scanner is being used instead of a terminal-aware line editor like JLine.
• The parser handles a useful subset of shell syntax, not the full POSIX shell grammar.
• Redirection logic is partly duplicated inside EchoCommand.
• Some error handling intentionally fails silently instead of surfacing exact errors.
• Pipeline execution merges stderr into stdout for external commands.
• The shell does not currently track exit status as shell state.
• Features like history, tab completion, environment expansion, globbing, and job control are not implemented yet.
• Some docs in the repo are stale compared to the current codebase.

I actually think these limitations make the project more interesting, not less.

Why? Because they clearly show the next frontier.

Once I understood how to build:

• a parser
• a built-in command system
• a process launcher
• a pipeline engine
• redirection support

the missing features stopped feeling mysterious. They became concrete engineering tasks.

If I Were Rebuilding This from First Principles

If I had to rebuild this project from scratch today, I would follow almost the same sequence:

1. Start with a basic REPL

First, I would build a loop that:

• prints a prompt
• reads a line
• exits on EOF

Until this works, nothing else matters.

2. Add a tokenizer before adding commands

I would not rush into implementing cd or pipelines immediately.

The parser is the foundation. If tokenization is sloppy, every later feature becomes fragile.

3. Add built-ins that require shell state

The first built-ins I would add are:

• exit
• pwd
• cd

These force the architecture to acknowledge that some commands must run inside the shell process.

4. Add external command execution through ProcessBuilder

Once built-ins work, external execution becomes much easier to reason about because the dispatch boundary is clear:

• built-in -> run in JVM
• external -> spawn process

5. Add redirection as parser output, not ad hoc string handling

I would absolutely keep the same idea of converting >, >>, 2>, and 2>> into internal markers early in parsing.

That simplification pays for itself immediately.

6. Add pipelines only after commands can work with streams

Pipelines are easiest once commands can conceptually read from one stream and write to another.

That is exactly why having executeInPipeline(...) was the right abstraction.

Why Java Was an Interesting Choice

Most people associate shell-building projects with C, Rust, or Go.

Doing it in Java was interesting because it forced me to think carefully about where the JVM helps and where it gets in the way.

Java gave me:

• strong structure through classes and interfaces
• easy file and path handling
• ProcessBuilder for child process execution
• a clean testing story with JUnit
• simple stream abstractions for piping data

But Java also made some things more explicit:

• there is no native fork() model like in C
• interactive terminal handling needs extra tooling like JLine
• true shell semantics still have to be implemented manually

That tradeoff was valuable. It made me focus less on raw syscalls and more on architecture, parsing, streams, and process orchestration.

The Biggest Takeaways from This Project

This project changed how I think about the terminal.

Before this, a shell felt like a black box sitting between me and the OS.

Now I think of it as a layered system:

• there is a prompt
• there is a parser
• there is a command dispatcher
• there is shell-owned state
• there are built-ins
• there are child processes
• there are streams connecting processes together
• there are file descriptors and redirections changing where output goes

That shift is the real value of the project.

Yes, I built something that runs commands.

But more importantly, I built a mental model for how real shells work.

Conclusion

Building my own shell in Java was one of the best ways I could have learned systems fundamentals. It forced me to think about parsing, state management, process creation, and stream orchestration.

If you want to understand shells beyond surface-level command usage, I highly recommend building one yourself. There is no better way to stop seeing the terminal as magic.

References and Further Reading

These are the sources I used to cross-check shell behavior and Java process execution while documenting the project:

• POSIX Shell Command Language: https://pubs.opengroup.org/onlinepubs/9799919799/utilities/V3_chap02.html
• POSIX historical shell command language reference: https://pubs.opengroup.org/onlinepubs/7908799/xcu/chap2.html
• Java ProcessBuilder documentation: https://docs.oracle.com/en/java/javase/25/docs/api/java.base/java/lang/ProcessBuilder.html
• Java Process API guide: https://docs.oracle.com/en/java/javase/20/core/process-api1.html
• CodeCrafters announcement for the shell challenge in Java: https://forum.codecrafters.io/t/build-your-own-shell-now-supports-c-c-java/628

Source Code Map

For anyone reading this blog alongside the project, these are the most important files:

• src/main/java/Main.java
• src/main/java/com/adi/shell/core/Shell.java
• src/main/java/com/adi/shell/core/ShellContext.java
• src/main/java/com/adi/shell/parser/CommandParser.java
• src/main/java/com/adi/shell/command/Command.java
• src/main/java/com/adi/shell/command/CommandRegistry.java
• src/main/java/com/adi/shell/command/ExternalCommandExecutor.java
• src/main/java/com/adi/shell/pipeline/PipelineExecutor.java
• src/main/java/com/adi/shell/command/builtin/EchoCommand.java
• src/main/java/com/adi/shell/command/builtin/ExitCommand.java
• src/main/java/com/adi/shell/command/builtin/TypeCommand.java
• src/main/java/com/adi/shell/command/builtin/PwdCommand.java
• src/main/java/com/adi/shell/command/builtin/CdCommand.java
• src/main/java/com/adi/shell/io/StreamUtils.java

And if you want to understand how I validated the implementation, the test suite is worth reading too:

• src/test/java/com/adi/shell/parser/CommandParserTest.java
• src/test/java/com/adi/shell/pipeline/PipelineExecutorTest.java
• src/test/java/com/adi/shell/command/ExternalCommandExecutorTest.java
• src/test/java/com/adi/shell/command/builtin/*.java
• src/test/java/com/adi/shell/core/*.java
• src/test/java/com/adi/shell/io/StreamUtilsTest.java