2023 Year in Review

In this blog post, we look back on the year and overview the highlights of 2023 for the Job Dispatcher team at EMBL-EBI.

In 2023, the Job Dispatcher team launched a brand new frontend available at https://www.ebi.ac.uk/jdispatcher. This website reorganises the tool and documentation pages and adds new features. Among the new features are: a new landing page, a new ‘Your Jobs’ page, improved tool webforms, improved result pages, and interactive visualisations available throughout the result pages of multiple sequence alignemt and sequence similarity search tools.

Highlights of the year

The main focus of the Job Dispatcher team during 2023 has been on

• the launch of JD’s brand new frontend
• the launch of JD’s new documentation
• development of the CI/CD pipelines for deploying the new frontend

There has been a clear commitment to improving the visibility of both the team and the Job Dispatcher services offerings.

The successful launch of the new angular frontend in production, is a major achievement for the Job Dispatcher team in 2023.

Service updates

Sequence analysis tools running under JD are categorised according to their functionality and have been regularly updated to their latest available versions. Sequence dataset updates and releases are routinely deployed for datasets such as UniProt, PDBe, Ensembl Genomes, WormBase Parasite and ENA. For more information about the state of our datasets, have a look at our latest post: Dataset updates for November 2023, and previous blog posts.

In 2023, more than 109 million JD jobs were performed on EMBL-EBI’s high-performance computing clusters. This corresponds to ~28 million additional jobs, when compared to 2022, and corresponds to ~9 million jobs per month and ~2 million per week. The majority of these jobs (~94%), were launched programmatically, with only about ~6% of jobs submitted through interactive web interfaces. Notably, there were 977 thousand unique users that ran JD jobs in 2023; an increase of 110 thousand from 2022.

Training and outreach

The team continues its involvement with support, training and outreach. In 2023, we triaged more than one thousand user support and feedback requests, and participated in several EMBL-EBI training events and successfully organised the third edition of the Job Dispatcher two-day workshop. See below a list of events in which the team has participated:

• Computational Biology Training in Hematology 2023 - “Code development and code release - best practices”

• Primer for Predocs 2023 - “Introduction to EBI web services”

• Protein Biology Course - “A general introduction to programmatic access”

• Webinar - “A guide to sequence analysis resources in Job Dispatcher”

• Structural Bioinformatics - “Sequences: Alignments and Annotation Live Q&A”

• Job Dispatcher Course 2023 - “Exploring EMBL-EBI sequence analysis tools and managing bioinformatics workflows”

The team has presented a poster at ISMB/ECCB 2023 Lyon:

Madeira F, Lee J, Madhusoodanan N, Tivey ARN, Eusebi A, Niewielska A, Butcher S. Introducing the new Job Dispatcher website: new features, user documentation, blogs and more! [version 1; not peer reviewed poster]; doi: m9.figshare.23937726.v1

Concluding remarks

The authors wish to acknowledge past members of the team, as well as systems infrastructure and web administrators for their continued support. We would like to also thank all EMBL-EBI service teams for their invaluable help in providing biological data, applications and expertise.

In retrospect, 2023 presented both challenges and rewards for the Job Dispatcher team. Looking ahead to 2024, we are enthusiastic about supporting the research community at EMBL-EBI and around worldwide.