Jupyter Notebook: A Complete Guide for Data Science

Jupyter Notebook is an open-source web application that lets you create and share documents containing live code, equations, visualizations, and explanatory text. It has become the standard environment for data science, scientific computing, and exploratory programming.

Installing Jupyter

# With pip
pip install jupyter

# With conda (recommended for data science)
conda install jupyter

# Launch
jupyter notebook

Running jupyter notebook opens a browser window showing the notebook dashboard — a file browser rooted in the directory where you ran the command.

JupyterLab vs Classic Notebook

JupyterLab is the next-generation interface. The classic notebook interface (jupyter notebook) is simpler but less extensible. JupyterLab offers a more IDE-like experience with drag-and-drop panels, a built-in terminal, and a file editor. Both run the same kernel and can open the same .ipynb files. For new projects, start with JupyterLab.

The Notebook Interface

A notebook is a sequence of cells. Each cell can be code, markdown, or raw text.

Code Cells

Code cells contain executable Python (or other kernel languages). Press Shift+Enter to run a cell and advance to the next one. The output appears directly below the cell:

import pandas as pd
import numpy as np

data = {
    "name": ["Alice", "Bob", "Charlie"],
    "score": [85, 92, 78]
---
df = pd.DataFrame(data)
df.mean()

Output:

score    85.0
dtype: float64

Markdown Cells

Markdown cells support formatted text with headings, lists, links, tables, images, and LaTeX equations:

# Heading 1
## Heading 2

This is **bold** and *italic* text.

- List item 1
- List item 2

1. Numbered item
2. Numbered item

Inline equation: $E = mc^2$

Block equation:
$$
\int_{-\infty}^{\infty} e^{-x^2} \, dx = \sqrt{\pi}
$$

Raw Cells

Raw cells contain unformatted text that is not executed or rendered. They are useful for including raw HTML or LaTeX for nbconvert export.

Essential Keyboard Shortcuts

Master these to work efficiently:

Shift+Enter    → Run cell and select next
Ctrl+Enter     → Run cell and stay
Alt+Enter      → Run cell and insert below
Esc, then:
  a            → Insert cell above
  b            → Insert cell below
  dd           → Delete cell
  m            → Convert to markdown
  y            → Convert to code
  r            → Convert to raw
  h            → Show all shortcuts

Cell Operations

In command mode (Esc), you can also:

• c — Copy cell
• x — Cut cell
• v — Paste cell below
• Shift+v — Paste cell above
• z — Undo cell deletion
• Shift+up/down — Select multiple cells for batch operations

Magic Commands

Magic commands are special commands that start with % (line magic) or %% (cell magic). They extend Jupyter’s capabilities.

Line Magics

%timeit sum(range(1000000))    # time a statement
%time sum(range(1000000))      # time a block
%who                          # list all variables
%whos                         # list variables with details
%hist                         # show command history
%lsmagic                      # list all magics
%pdb                          # auto-debug on exception
%load_ext autoreload          # auto-reload modules
%env                          # list environment variables

Cell Magics

%%time                       # time the entire cell
%%capture output             # capture cell output
%%writefile my_script.py     # write cell content to file
%%bash                       # run shell commands
%%html                       # render HTML
%%latex                      # render LaTeX
%%javascript                 # run JavaScript
%%prun                       # profile the cell

Profiling with %%prun

%%prun
# Profile the entire cell
import pandas as pd
df = pd.DataFrame(np.random.randn(1000, 100))
df.apply(lambda x: x.mean())

Data Visualization

Jupyter renders plots inline by default:

%matplotlib inline

import matplotlib.pyplot as plt
import seaborn as sns

# Generate data
data = np.random.randn(1000)

# Plot
plt.figure(figsize=(10, 6))
plt.hist(data, bins=30, alpha=0.7, color="steelblue")
plt.title("Distribution of Random Data")
plt.xlabel("Value")
plt.ylabel("Frequency")
plt.grid(True, alpha=0.3)
plt.show()

Interactive Widgets

from ipywidgets import interact, widgets

@interact(x=(0, 10, 0.1), y=(0, 10, 0.1))
def plot_sine(x=5, y=5):
    plt.figure(figsize=(8, 4))
    t = np.linspace(0, 10, 100)
    plt.plot(t, np.sin(x * t + y))
    plt.grid(True)
    plt.show()

Altair and Plotly for Interactive Charts

For richer interactive visualizations, use Altair (declarative statistical graphics) or Plotly (interactive charts):

import altair as alt
import pandas as pd

data = pd.DataFrame({
    'x': range(100),
    'y': np.random.randn(100).cumsum()
---)

alt.Chart(data).mark_line().encode(
    x='x',
    y='y'
).interactive()

These libraries produce HTML-based visualizations that support hover tooltips, zooming, and panning — directly in the notebook output.

Working with Data

Jupyter’s cell-by-cell execution is perfect for data exploration:

# Cell 1: Load data
import pandas as pd
df = pd.read_csv("sales_data.csv")

# Cell 2: Inspect
df.head()
df.info()
df.describe()

# Cell 3: Clean
df = df.dropna()
df["date"] = pd.to_datetime(df["date"])

# Cell 4: Analyze
monthly = df.groupby(df["date"].dt.month)["revenue"].sum()
monthly.plot(kind="bar")

Each cell shows its output immediately, so you can see the result of each step before moving to the next.

Notebook Kernels

Jupyter supports multiple programming languages through kernels:

# Install kernels
pip install ipykernel      # Python (default)
pip install jupyterlab-sql  # SQL
pip install rpy2           # R (with rpy2)
pip install julia          # Julia

Switch kernels from the Kernel menu in the notebook interface.

Sharing Notebooks

nbviewer

Upload your notebook to GitHub and view it at nbviewer.jupyter.org. The notebook renders as a static HTML page with all outputs visible.

Converting to Other Formats

# HTML (with all outputs)
jupyter nbconvert --to html my_notebook.ipynb

# PDF (requires LaTeX)
jupyter nbconvert --to pdf my_notebook.ipynb

# Markdown
jupyter nbconvert --to markdown my_notebook.ipynb

# Python script
jupyter nbconvert --to script my_notebook.ipynb

# Slides (reveal.js)
jupyter nbconvert --to slides my_notebook.ipynb --post serve

Voilà Dashboards

Voilà converts notebooks into standalone web applications:

pip install voila
voila my_notebook.ipynb

Voilà hides all code cells, showing only the outputs — making notebooks look like dashboards or reports.

JupyterLab

JupyterLab is the next-generation interface for Jupyter. It provides a more IDE-like experience with:

• Multiple panels (notebook, terminal, file browser)
• Drag-and-drop cell rearrangement
• Built-in file editor
• Extension system
• Better Git integration

# Install
pip install jupyterlab
jupyter lab

JupyterLab Extensions

The extension ecosystem dramatically expands JupyterLab’s capabilities:

# Install extensions via pip or the extension manager
pip install jupyterlab-git            # Git integration
pip install jupyterlab-lsp            # Code autocompletion
pip install jupyterlab-toc            # Table of contents
pip install jupyterlab-spellchecker   # Spell check
pip install jupyterlab-execute-time   # Show cell execution time

Best Practices

1. Keep notebooks linear — run cells top to bottom to avoid stale state
2. Use version control output — use jq or nbdime to diff notebooks
3. Name your cells — use markdown headings to structure long notebooks
4. Clear outputs before committing — use Cell → All Output → Clear or a pre-commit hook
5. Use %autoreload during development — auto-import changes from modules
6. Avoid heavy computations in notebooks — move production code to Python modules
7. Export to scripts for production — notebooks are for exploration, not deployment
8. Use a consistent kernel — pin your kernel version in the notebook metadata

FAQ

How do I reset a notebook without losing my code?

Use Kernel → Restart & Clear Output to reset the execution state. To keep outputs but reset variables, use Kernel → Restart & Run All.

Why is my notebook showing “Kernel Not Found”?

This typically means the kernel specification for your notebook’s language is not installed. Reinstall the kernel with python -m ipykernel install --user for Python, or install the appropriate kernel package for your language.

How do I handle large datasets in Jupyter?

Use chunked loading with pandas (chunksize parameter), sample the data during exploration, and move heavy processing to Python scripts. The %%time magic helps identify slow cells.

Can I use Jupyter for production ETL pipelines?

Notebooks are designed for exploration, not production pipelines. Export your code to Python scripts and use workflow managers like Airflow or Prefect for production ETL.

What is the difference between %matplotlib inline and %matplotlib notebook?

%matplotlib inline renders static PNG plots. %matplotlib notebook (or %matplotlib widget) enables interactive plots with zooming, panning, and resizing. For JupyterLab, use %matplotlib widget for the best interactive experience.

Jupyter Lab vs Classic Notebook

Jupyter Lab offers a more complete IDE-like experience compared to the classic notebook interface. It provides a file browser, multiple panes, a terminal emulator, and a debugger that can set breakpoints and inspect variables within notebook cells. The extension ecosystem in Jupyter Lab is more robust, with extensions for Git integration (jupyterlab-git), code formatting (jupyterlab-code-formatter), and interactive plotting (jupyterlab-matplotlib). The debugger frontend (jupyterlab-debugger) supports step-through debugging for both Python and R kernels. Variable inspector and data viewer panels let you examine DataFrames and arrays without print statements. Voilà converts notebooks into standalone dashboards that hide code cells, useful for sharing results with non-technical stakeholders. Jupyter Lab’s theming system supports dark mode out of the box, reducing eye strain.

Jupyter in Production and Collaboration

JupyterHub deploys notebooks for multiple users with authentication and resource isolation. Combine it with Docker spawner to give each user an isolated environment. Papermill parameterizes notebooks, executing them with different inputs from the CLI, enabling scheduled data pipeline execution. nbconvert exports notebooks to HTML, PDF, LaTeX, or Markdown with templates for custom styling. Review notebooks with ReviewNB or nbdime for meaningful diffs. Store notebooks in standard .ipynb format and strip output cells before committing to reduce repo bloat using jupyter nbconvert --ClearOutputPreprocessor.enabled=True as a pre-commit hook.

For a comprehensive overview, read our article on Advanced Git Commands.

For a comprehensive overview, read our article on Chrome Devtools Guide.