Conda Environments for Quarto Documents

How to specify a specific Conda environment when rendering quarto documents.

Creative commons license, created by Ralph

Introduction

This article sets out minimal instructions on rendering quarto documents that rely on specified conda virtual environments. This article collates information from:

• Quarto Documentation: Using Conda
• Quarto Documentation: Using Python
• nb_conda_kernels readme

It is presumed that you will be working within a git repository at times. If this is not the case, ignoring steps specifying git instructions should not affect your ability to successfully render the quarto documents.

A Note on the Purpose

The purpose of this article is not to explore reasons for using conda enviroments or to compare the pros and cons of the many different options for managing python virtual environments. It aims to help the reader configure quarto documents to run with a specified conda environment while remaining minimal, opting to link to sources of further information where discussion may complement the content.

Intended Audience

Python practitioners familiar with conda environment management [1] who are less familiar working with quarto documents [2].

The Scenario

You are writing a quarto document that contains python code. You would like to use conda to manage your python dependencies. You are encountering problems in having quarto select the appropriate conda environment.

Named Environments

This article covers having quarto execute with “prefix” conda environments. This setup may be useful specifically for a website where different site pages have different dependencies.

However, many readers may wish for a simpler solution. It is possible to have quarto websites and documents execute with a named environment instead. If you have an environment created like below:

conda create -n my-awesome-env python -y

Then including the following statement within either the _quarto.yml or quarto document’s YAML header should be enough to guarantee that the target environment is picked up when rendering:

jupyter: my-awesome-env

Additionally, if you would just prefer the site or document to render with whatever version of python is available in the currently active environment, then use:

jupyter: python3

Many thanks to Ethan for this tip.

What You’ll Need:

• Conda or miniconda
• Quarto
• Text editor eg VS Code
• Python package manager (eg pip)
• Access to a command line interface (CLI) such as terminal / Bash.
• requirements.txt:

requirements.txt

nbclient
nbformat
palmerpenguins

• git (optional)

Configuring Quarto with Conda Env

Project Structure

1. Create a new project folder. Open a terminal and change directory to the new project.
2. Save the requirements to a requirements.txt file.
3. Create a new quarto document in the project root. In VS Code, use
   File New File... Quarto Document.
4. Write the following content to a python code chunk in the quarto file and save as penguins.qmd

penguins.qmd

df = penguins.load_penguins().dropna()
df.head()

Configure the Environment

5. Create a new conda environment with the -p flag and give it a suitably descriptive name ¹. Ensure that the environment is built with python 3.11 ².

CLI

conda create -p SOME_ENV_NAME python=3.11 -y

6. Activate the environment.

CLI

conda activate ./SOME_ENV_NAME

7. Install the requirements file.

CLI

pip install -r requirements.txt

8. Add a .gitignore file and include the name of the local environment directory created in step 4.

.gitignore

SOME_ENV_NAME/

Configure the Quarto Project

9. Create a _quarto.yml configuration file in the project root. In this file, we will specify that the quarto render command should render any qmd files and ignore any files found within your local environment. Add the following content:

_quarto.yaml

project:
  type: website
  output-dir: docs
  render:
    - "*.qmd"
    - "!/./SOME_ENV_NAME/"

10. Use conda to install the nb_conda_kernels package. This is used to manage python jupyter kernels for notebooks.

CLI

conda install nb_conda_kernels

11. Copy the path returned from the below command

CLI

jupyter --config-dir

12. Create a jupyter_config.json in the jupyter config directory:

CLI

touch <INSERT_YOUR_CONFIG_DIR>/jupyter_config.json

13. Write the below content to this file and save.

CLI

echo -e "{\n  "CondaKernelSpecManager": {\n    "kernelspec_path": "--user"\n  }\n}" >> <INSERT_YOUR_CONFIG_DIR>/jupyter_config.json

14. Run the below command to return a list of available kernels:

CLI

python -m nb_conda_kernels list

15. Copy the name (not the path) for the environment that you created with the format conda-env-<YOUR_ENV_NAME>-py.
16. Open penguins.qmd. Adjust the YAML header so that it contains the following:

penguins.qmd

jupyter: 
  kernelspec:
    name: "conda-env-<YOUR_ENV_NAME>-py"
    language: "python"
    display_name: "<YOUR_ENV_NAME>"

17. You should now be able to render the quarto project, confirming that the target environment was activated in the CLI output. eg:

CLI

quarto render

Starting <YOUR_ENV_NAME> kernel...Done

Tips

• When encountering issues with quarto render, it can be informative to examine the output of quarto check or quarto check jupyter in the CLI.
• As there are many steps to configuring conda, it may be a good idea to create a dedicated conda environment for all of your quarto projects. Quarto attempts to select an appropriate kernel based upon the content of the first executable python code chunk in your quarto document. Usually, this chunk would contain the import statements. However, over time this would likely result in package conflicts over time.
• The approach set out in this how-to would be a good fit for a website built with quarto, where the configuration steps can be performed only once in a parent website environment, and then specific, minimal environments created for each article requiring a python environment.
• Alternatively, consider using venv or poetry to manage python environments [3] for quarto projects.

Troubleshooting

• You’ve created a new environment and it is not discovered when running python -m nb_conda_kernels list:
  • Activate your new environment
  • pip install ipykernel
  • Run:

  python -m ipykernel install --user --name <INSERT_ENV_NAME> --display-name "<INSERT_DISPLAY_NAME>"

  • Deactivate the new environment.
  • Run python -m nb_conda_kernels list once more and the new env should appear.
  • Taken from this SO thread

Conclusion

This article has walked the reader through setting up a basic quarto project, creating a conda environment, and configuring a quarto document to render with a specified environment. For more help with quarto, consult the quarto-cli GitHub repository [4] and the RStudio Community [5] (soon to rebrand to the Posit Community).

fin!

References

[1]

Anaconda, Inc., “Conda documentation - Managing environments.” https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html

[2]

Quarto, “Hello Quarto.” https://quarto.org/docs/get-started/hello/rstudio.html

[3]

Alex K. Gold, “Conda documentation - Managing environments.” https://do4ds.com/chapters/sec1/1-1-env-as-code.html#the-package-layer

[4]

quarto-dev, “Quarto CLI GitHub repo.” https://github.com/quarto-dev/quarto-cli/issues

[5]

Posit Software, “Posit Community.” https://community.rstudio.com/

Footnotes

1. When creating conda environments, the use of generic names such as env will result in conda prepending the environment name with numbers to avoid conflicts. Use descriptive environment names in order to avoid this, eg penguins-env.

2. nb_conda_kernels (a package required in a later step) does not currently work with python 3.12 or newer.