QHub HPC

QHub HPC is an opinionated open source deployment of jupyterhub based on an HPC jobscheduler. QHub HPC is a “distribution” of these packages much like Debian and Ubuntu are distributions of Linux. The high level goal of this distribution is to form a cohesive set of tools that enable:

• environment management via conda and conda-store

• monitoring of compute infrastructure and services

• scalable and efficient compute via jupyterlab and dask

• deployment of jupyterhub on prem without requiring deep devops knowledge of the Slurm/HPC and jupyter ecosystem

Features

• Scalable compute environment based on the Slurm workload manger to take advantage of entire fleet of nodes

• Ansible based provisioning on Ubuntu 18.04 and Ubuntu 20.04 nodes to deploy one master server and N workers. These workers can be pre-existing nodes in your compute environment

• Customizable Themes for JupyterHub

Table of Contents

Contents:

• Overview
  • All Nodes
  • Master Node
  • Worker Nodes
• Conda
  • Adding Conda Environments
  • Updating a conda environment
• Ansible
  • Creating Ansible Inventory
• Benchmarking
  • Storage
  • Network
• ContainDS Dashboards
  • Necessary Steps to use ContainDS Dashboards with Qhub-HPC
  • Restarting a Stopped Dashboard
• Dask
• JupyterHub
• Slurm
  • Checking Health of Slurm Cluster
  • Get current job queue including running jobs
  • Getting information about a job
  • Configuring/Adding Node Information
  • Modifying Node State
  • Node States
  • Adding Slurm Partitions
• Backup
  • Configuration
  • Manual Backup
• Frequently Asked Questions
  • Why on-prem hpc instead of kubernetes?

Indices and tables

• Index

• Module Index

• Search Page