Python
Using uv
One way to use Python on the cluster is to use uv as packages and project manager.
Warning
This section only applies to pure Python projects, ie projects which only use Python packages as dependencies.
Install uv on the cluster
You can install uv yourself directly on your cluster account by running the following:
Then add the following to your ~/.bashrc in order to avoid cluttering your home directory (replace <user> with your login):
export UV_CACHE_DIR=/beegfs/data/<user>/.cache/uv
export UV_PYTHON_INSTALL_DIR=/beegfs/data/<user>/.uv/python
export UV_TOOL_DIR=/beegfs/data/<user>/.uv/tools
After logging out and reconnecting to the cluster, the uv and uvx commands should be available.
Setup your project locally
Setup your project locally on your machine using uv:
- Init a project with
uv init - Select a python version with
uv python pin - Add dependencies with
uv addor by editingpyproject.toml
For more information about uv usage, see the https://docs.astral.sh/uv/ or our training slides.
Run your project on the cluster
If you want to run a project (say, a python script script.py) on the cluster, you just have to transfer your data in your /beegfs/data/user directory, and then run (preferably via slurm):
uv will take care of installing the correct Python version, the package dependencies, and running your script.
Troubleshooting
Illegal instruction during python import
If running import polars or another package raises an Illegal instruction or Instruction non permise, it may be because the current node CPU doesn't support the required optimizations.
polars for example requires several optimization flags like avx2, which are available on intel CPU starting from skylake architecture (so skylake, and maybe amd ?).
You can ask for specific CPU architectures with the SLURM parameter --constraint, something like:
To see constraints for every cluster node, you can do: