Conda Exists Because Scientific Python Is Not Python

Why pip alone cannot install the numerical stack reliably, what conda does about it, what it costs, and where uv and pixi have moved the line since. As of early 2025.
Python
Tooling
Author

Ravi Kalia

Published

February 2, 2025

Conda

Conda attracts complaints: it is slow, it is heavy, it fights with pip. Most of the complaints are true. It still earns its place in scientific computing, and the reason is one fact about the software: the libraries that make Python fast are not written in Python. NumPy, SciPy, PyTorch and their relatives are thin Python skins over C, C++, Fortran and CUDA, and a package manager that only understands Python packages cannot install them reliably. This post says what pip cannot do, what conda does instead, what it costs, and, in a closing note added later, where the newer tools have moved the line. I gave a talk on this, and the post is the argument from it.

pip installs Python packages, and the hard part is not a Python package

A pip wheel can carry compiled code, and for the common platforms the numerical libraries publish wheels that work. What a wheel cannot express is a dependency on something outside Python: a specific BLAS, a compiler runtime, CUDA and cuDNN, a system library like HDF5 or GDAL, or R next to Python in the same environment. When two wheels each bundle their own copy of such a library, they can disagree, and the failure is a segmentation fault at import time rather than a version error at install time. pip also manages one Python at a time: switching interpreter versions is someone else’s job (pyenv, the system), and a system Python is shared with the operating system, so an upgrade for one project can break a tool the OS relies on.

Languages such as Node and Ruby get by with their own package managers because their ecosystems rarely reach below the language runtime. Scientific Python reaches all the way to the chip.

conda manages environments of binaries, in any language

Conda treats the interpreter, the compiled libraries and the Python packages as the same kind of thing: packages, built once for each platform and installed as binaries into an isolated environment. That is what lets one environment pin Python 3.9, R 4.2, a particular BLAS and CUDA 11 together and reproduce the same set on a colleague’s machine. Three parts of the design carry the weight:

  • Binary packages with their non-Python dependencies declared, so that the BLAS NumPy links against is an explicit, versioned package rather than whatever is on the machine.
  • Channels: repositories of those packages. conda-forge is the community-built one and covers most of scientific computing; bioconda adds bioinformatics; vendors run their own for PyTorch, NVIDIA and Intel builds. Channel priority decides which wins.
  • A solver that finds a set of package versions satisfying every constraint at once, across languages, and refuses when none exists, rather than installing the last one requested and leaving the conflict to surface at runtime.

The daily commands are short:

conda create -n myenv python=3.9
conda activate myenv
conda install -c conda-forge numpy pandas scipy
conda env export > environment.yml     # the reproducible record

Four habits keep it out of trouble. Use conda-forge as the primary channel and add specialised channels only when a package needs them; the Anaconda defaults channel carries licence terms that matter for commercial use. Install with conda first and pip only for packages conda-forge lacks, because pip does not tell conda’s solver what it did. Expect the solve to take a while on a large environment, and reach for mamba, a drop-in reimplementation of the solver, when it takes too long. And commit the environment.yml, since the environment on disk is not the record.

The cost is real and worth paying for one kind of work

Conda environments are large, installs are slower than pip, the solver can take minutes, and the pip-inside-conda seam is a recurring source of breakage. For a web service written in pure Python none of that buys anything, and pip with a lockfile is the better tool. For a project whose dependencies reach compiled libraries, GPUs or a second language, and which has to run on a collaborator’s machine next year, the cost buys the one thing pip cannot offer: a reproducible environment down to the binaries.

Where it stops holding, and what has moved since

Two things have shifted the line since this was written in early 2025. uv made the pure-Python case fast and locked, so the “pip with a lockfile” option above is now the default for anything that does not need compiled dependencies beyond what wheels carry. And pixi put a proper lockfile and a fast solver over the same conda-forge packages, which removes most of conda’s own friction while keeping its reach. Numerical computing reproducibility sets out where each tool stops on the ladder from source code down to the chip, and is the current view; this post is the case for why that ladder has more rungs than pip can see.

Dependencies. Fight. Conda. Referees. Slower. Installs. Buy. Reproducible. Science. Worth. It.

References