Features

This page covers user-facing features added to bwa-mem3 on top of upstream bwa-mem2. None of these features change default behavior: output produced by bwa-mem3 mem without any of these flags is byte-identical to the corresponding bwa-mem2 output (except for the @PG ID: and PN: fields which now read bwa-mem3).

--meth bisulfite alignment mode (PR #13)

--meth adds native bisulfite/EM-seq alignment to bwa-mem3 index and bwa-mem3 mem in a single binary — no Python, no separate post-processing step, no bwameth.py dependency. In the default --meth-scoring collapsed mode it reproduces bwameth.py’s read placement (a placement drop-in, not a byte-for-byte clone); --meth-scoring genomic opts into variant-aware scoring bwameth cannot produce.

bwa-mem3 index --meth ref.fa          # once per reference
bwa-mem3 mem --meth ref.fa R1.fq R2.fq | samtools sort -o out.bam

index --meth builds a dual index: the normal index over the original reference plus a converted seed index <ref>.meth.* (over <ref>.meth.fa, a doubled reference with f-prefixed C→T and r-prefixed G→A contigs). Reads seed in that 3-letter space but are scored against the original 4-letter reference, so the index layout is not the same as bwameth.py’s single .bwameth.c2t reference.

mem --meth projects each read (R1 C→T, R2 G→A) to find seeds in the .meth index, preserving the original bases on the first-class bseq1_t.meth_orig_seq field (a YS:Z/YC:Z comment carrier is a fallback only; neither reaches the BAM). It then extends and scores against the original 4-letter reference with a per-strand asymmetric matrix, consolidates the f/r contig pairs back to one @SQ per real chromosome, emits Bismark-compatible XR:Z (read conversion direction), XG:Z (genome strand), and XM:Z (per-base methylation call string) auxiliary tags, restores the original bases into the BAM SEQ field for CpG-calling tools, optionally applies a chimera QC heuristic (longest M/=/X run < 44% of read length → set 0x200, clear proper-pair 0x2, cap MAPQ at 1) when --chimera-qc is passed (off by default), and writes a @PG ID:bwa-mem3-meth entry.

In the default collapsed mode this reproduces bwameth.py’s read placement (chrom, pos) for the standard case, while scoring against the original reference rather than in collapsed space — so it is not byte-for-byte identical to bwameth output. Stacks on PR #12 (--bam). See the Methylation Reference for full details.

Vendored mimalloc allocator (PR #19)

bwa-mem3 vendors mimalloc v3.3.0 as a pinned submodule at ext/mimalloc and links it into every binary by default (USE_MIMALLOC=1). On Linux, static linkage uses --whole-archive; on macOS, dyld-interposed shared linkage is used.

Measured on AWS c7g.4xlarge (Graviton3, 16 threads, 29M 150 bp paired-end exome-capture reads vs hg38, page cache dropped between iterations): −24.5% wall-clock time (528.6 s → 424.7 s) compared to the same build with USE_MIMALLOC=0. No user-visible interface change; no runtime configuration required.

USE_MIMALLOC=0 is a supported best-effort opt-out and is CI-gated on Linux x86. bwa-mem3 version prints the mimalloc version string when it is active.

--supp-rep-hard-cap supplementary MAPQ rescoring (PR #56)

Supplementary alignments for a split read inherit MAPQ from the full-read scoring pipeline. Competing repetitive chains for the supplementary fragment are filtered out during full-read chain scoring (mem_chain_flt) before Smith-Waterman, so they never contribute to sub/sub_n. A supp fragment landing in a CCATCC repeat that would map equally well to 50+ locations standalone can therefore carry MAPQ=60 from its primary.

--supp-rep-hard-cap INT opts into rescoring: if any seed in a supplementary alignment’s chain has >=INT genome occurrences (from the SMEM SA count), the supplementary MAPQ is forced to 0. Primary alignment MAPQ and coordinates are unaffected. Default output (no flag) is byte-identical to upstream bwa-mem2.

The SMEM SA-occurrence count is preserved on each seed as mem_seed_t.n_hits and propagated to mem_alnreg_t.chain_n_hits during chain-to-alignment conversion. Typical values for INT are 5–20; lower is more aggressive. The upstream bwa-mem2#260 reporter case drops from MAPQ=60 to MAPQ=0 at --supp-rep-hard-cap 18. Closes issue #46.

Shared-memory index: bwa-mem3 shm (PR #65)

bwa-mem3 mem reloads the FM-index from disk on every invocation. For hg38 the index is ~18 GB; for short alignment jobs (targeted panels, small sample batches) this load cost dominates runtime and makes per-invocation IOPS the bottleneck.

PR #65 ports the bwa shm command from bwa-mem v1 to bwa-mem3 with strict v1 CLI parity:

bwa-mem3 shm <index-prefix>    # load index into shared-memory segment once
bwa-mem3 mem <index-prefix> ...  # subsequent runs attach instead of re-reading
bwa-mem3 shm -d <index-prefix>  # detach and free the segment

The index lives in a POSIX shared-memory segment. Multiple bwa-mem3 mem processes on the same host share the same in-memory copy. Closes issue #64.

Warning — Stale index

bwa-mem3 shm does not detect when the on-disk index has been rebuilt. Always run bwa-mem3 shm -d <prefix> before running bwa-mem3 index and then re-stage with bwa-mem3 shm <prefix>. Using a stale shared-memory segment produces silently wrong alignments.

bwa-mem3 shm --meth (PR #67)

bwa-mem3 mem --meth <prefix> locates the .meth seed index built by bwa-mem3 index --meth <prefix> automatically. Before PR #67, staging a methylation index in shared memory required passing the full suffixed seed-index path to shm while continuing to pass the plain prefix to mem. The mismatch was easy to forget, and the failure mode — a run that silently attached the wrong segment — was difficult to diagnose.

PR #67 adds --meth support to bwa-mem3 shm so the same plain-prefix convention works end-to-end:

bwa-mem3 shm --meth ref.fa       # stages ref.fa.meth.*
bwa-mem3 mem --meth ref.fa ...   # attaches automatically
bwa-mem3 shm -d --meth ref.fa   # detaches

HN:i hit count tag (PR #42)

Every primary SAM/BAM record now carries an HN:i:<n> tag reporting the number of secondary alignment candidates clustered with this primary under XA_drop_ratio. This count is captured before the -h/max_XA_hits cap truncates the XA:Z: string, so HN reports the true number of alternate loci even when no XA:Z: field appears in the record.

This makes it possible to distinguish:

• HN:i:0 + no XA:Z: — genuinely unique mapper.
• HN:i:N + XA:Z:... (N ≤ -h) — multi-mapper with all alternates listed.
• HN:i:N + no XA:Z: (N > -h) — multi-mapper whose alternates were suppressed by the cap.

Motivated by lh3/bwa#438, which adds HN to bwa aln. HN is emitted in both SAM (mem_aln2sam) and BAM (mem_aln_to_bam) paths and is absent when -a (MEM_F_ALL) is active.

--bam=LEVEL direct BAM output (PR #12)

bwa-mem3 mem --bam (or --bam=0 through --bam=9) emits BAM directly via htslib, bypassing the SAM-text-to-BAM conversion round trip that normally occurs when the output is piped to samtools view -bS.

• --bam / --bam=0: uncompressed BAM (BGZF framing only) — near-zero CPU overhead, smaller than SAM text, fast downstream parsing.
• --bam=1..9: BGZF deflate at the specified level.
• No flag: SAM text on stdout (default, unchanged).

The implementation adds src/bam_writer.{h,cpp}, a new module that converts mem_aln_t to bam1_t via mem_aln_to_bam. htslib v1.21 is pulled in as a submodule at ext/htslib. On the bwameth.py example fixture (92,961 records), samtools view of --bam output vs SAM text produces a zero-line diff across all 11 SAM columns and all aux tags. See Best Practices → Output format for the recommended pipeline.

--smem-dedup SMEM deduplication

--smem-dedup opts into removing fully-identical duplicate SMEM seeds before SA expansion. Off by default → output byte-identical to baseline. When enabled, duplicate SMEMs (same rid, query span [m,n), SA interval [k,l) of size s) that appear adjacent in the sorted SMEM array are compacted in O(n) with no allocation.

Duplicate SMEMs arise because the FM-index is a B-tree and can contain duplicate keys, particularly in repeat-dense regions. On 50 k WGS reads vs hg38, roughly 10 % fewer SA lookups are performed with --smem-dedup active. The wall-clock impact on an arm64 host is ~1–2 % (seeding is a fraction of total runtime); on x86 workloads where SA expansion is a larger share the gain is correspondingly larger.

The accuracy impact is small and bounded: only reads that carried duplicate SMEMs can be affected, and only if the deduplication changes which chain wins. On the 50 k validation set, 2 reads (0.004 %) differed — one XS tag update on a MAPQ-60 read (primary coordinates unchanged) and one equal-score MAPQ-0 tie-break shift. Zero uniquely-mapped reads changed. See the root cause analysis for the full characterization.

Not byte-identical

Do not enable in pipelines that compare output to a bwa-mem2 or bwa-mem3 baseline. The changes are benign and bounded, but they are real SAM changes.

--min-ext-len short-seed extension filter

--min-ext-len INT opts into skipping banded Smith-Waterman extension of short seeds (< INT bp) that sit in a chain with a longer anchor seed — the anchor’s extension already covers them, so their own extension is redundant. Off by default (0) → output byte-identical to baseline.

Smith-Waterman extension is ~60 % of bwa-mem3 mem CPU, and almost all of it is spent on short seeds: seeds ≤40 bp hold roughly 90 % of all banded-SW cells yet are ~99 % wasted, because long seeds already resolve via the ungapped fast-path at near-zero cost. The filter drops those redundant short seeds before extension (mem_chain_drop_short_seeds, a stable in-place compaction of each chain’s seeds, called from mem_flt_chained_seeds in src/bwamem.cpp), so their extension never runs while seeding and chaining are untouched.

Recall-safe by construction. A chain whose seeds are all short is left intact — dropping its only evidence would unmap the read — so the filter never empties a chain. (An earlier version dropped every short seed unconditionally, which silently unmapped low-mappability reads: a 151 bp low-mappability sample lost 63 % of its mappings. The anchor guard fixes this; it is a strict recall improvement that can reduce work but never lose a read.)

Measured single-thread on hg38 (HG002 1M PE WGS, non-emptying filter):

• ~10 % lower main_mem CPU at --min-ext-len 30 (−9.5 % measured), with no recall loss — mapped count is identical to default (99.75 %). ~0.10 % of reads change locus (down from ~0.40 % under the old emptying filter), confined to the low-confidence tail; ~0.005 % of reads change at MAPQ ≥ 60.
• Higher thresholds no longer cliff: mapped count and ~0.10 % divergence hold flat across 30–50, because all-short chains are now protected.
• The previously-documented high-error F1 cliff and the cross-architecture speed figures were measured under the emptying behavior; both need a fresh bwa-mem3-bench run under the non-emptying filter. Indels and structural variants were never a contraindication (an indel leaves two still-long exact segments the fast-path handles).

30 is the recommended opt-in value. Because the speedup thins the extension stage — not seeding — the wall-clock gain is smaller than the cell-count reduction suggests (seeding, which the filter does not touch, dominates runtime). See CLI → mem --min-ext-len and Settings profiles for the recommended operating point.

--seed-order seed reordering before chaining

--seed-order <mode> reorders each read’s SA-resolved seeds before chaining. The default off preserves byte-identical output. The recommended opt-in mode is local-longest, which sorts seeds by decreasing length so the longest seed anchors its chain first and absorbs contained shorter seeds — those sub-seeds then never reach banded Smith-Waterman.

bwa-mem3 mem --seed-order local-longest -t 16 ref.fa R1.fq.gz R2.fq.gz | samtools sort -@ 4 -o out.bam -

Measured on 50,000 real WGS reads (1000 Genomes HG00096, hg38), local-longest reduces extended seeds by ~8.9 % (absorbed fraction increases from 38.2 % to 43.7 %). Since Smith-Waterman extension is typically the dominant per-read cost in bwa-mem3 mem, this translates to a meaningful throughput gain on extension-heavy workloads.

--seed-order local-longest is not byte-identical to the default — it can shift secondary alignments, XA:Z:, XS:i, HN:i, and a small number of primaries. Accuracy is flat on easy simulated data (holodeck, F1 ~94.4 %; no regression vs off), but hard-data F1 validation on divergent/indel-rich reads and GIAB benchmarks is not yet complete. For that reason, all non-off modes are opt-in only and the default stays off.

See Optimization checklist → Reorder seeds longest-first and Equivalence → Seed ordering for full details.

Changes catalog

See also: Methylation Reference → Overview · User Guide → Memory allocator · User Guide → Output: SAM/BAM, headers, tags · Getting Started → Quick start: shared-memory index · Best Practices → Output format