Notebook

Bioinformatics Twitter is having...a moment. What began as one man's exasperated rant against poorly documented and distributed code has bloomed into a protracted debate about what constitutes "good" bioinformatics software and which kind(s) of interfaces developers should be expected to provide if they "really" want people to use their software. For those that have been around a while …

I contend that you should be treating your solo software projects like a collaborative project and using pull requests, an issue tracker, and other social development tools.

A few comments on the new "pipe" output flag in Snakemake.

A primer on software testing for scientists and researchers.

About a month ago, I was intrigued to see a bit of Twitter activity around a new bioRxiv preprint. The manuscript describes HULK, a new bioinformatics tool that implements some useful comparison metrics and operations for analyzing (meta)genomes. HULK is based on a new algorithm called histogram sketching (HistoSketch for short), following the trend of related sketching algorithms (HyperLogLog …

One of the fundamental insights of the kevlar de novo variant caller is the framing of the variant discovery problem as a search for novel k-mers. In this case, "novel" means abundant in the focal sample and effectively absent from all control samples. In the early stages of creating kevlar, it quickly became clear that many k-mers satisfying these simple …

BAM files with sequence alignments sorted by genomic position seem to be the new currency of exchange for large-scale human genome sequencing projects. This is convenient and practical in many ways for many people. But in my current research I work a lot with tools that only want/need the sequence information and, for whatever reasons, support only FASTA or …

Keeping track of k-mers for simple operations has become a fundamental component of many bioinformatics techniques. Two common operations on k-mers include set membership queries ("is k-mer X present in data set Y?") and abundance queries ("how many times does k-mer X occur in data set Y"). Several probabilistic data structures have been developed to support …

In which I explore read alignments in SAM format and discuss the pros and cons of various approaches to distinguishing perfect matches from imperfect matches.

There are many wonderfully elegant and efficient tools for performing all sorts of exact and inexact searches on large collections of DNA sequences. Experience has shown, however, that these tools are usually very rigid with respect to their assumptions about input data. If input files are compressed in a certain way, or stored in a non-standard format, there is usually …