fiction
- “WPM” (2003)
- “Incidental Music” (2005)
- “Footnotes” (2006)
- “The perpetual now” (2011)
- "The Obscure Journey" / "Le voyage obscur" (2013)
- English: in Winter Journeys
- French: in Le voyage d'hiver et ses suites and in La Bibliothèque Oulipienne no. 215
- "Nobody In Particular" in Bookstories (2014)
non-fiction
- “T-314” (2005)
- “Possession” (2005)
- “Let me ride” (2011)
- Many Subtle Channels: In Praise of Potential Literature (2012)
- Excerpt: “La Bibliothèque Impossible” in The Believer
- Excerpt: “Little Demons of Subtlety” at Berfrois
- "99 Preparatory Notes to Experimental Literature" (2018)
- "99 Preparatory Notes to the Work of François Morellet" (2019)
- What's Good: Notes on Rap and Language (2022)
not fiction but not very informative either
- Selected two-word phrases (2001-present)
- Some words that rhyme more or less with chronic (2003)
- On a Faulkner quote (2003)
- “What's in a band name? Everything!” (2004)
- Beau présent solipsiste (2007)
- The Heath Ledger Memorial Party Game (2007-2009)
- Une morale élémentaire (2008)
- Manifesto (2008)
- Index of first lines (2008)
- Index des premiers vers (2009)
- Prose morale élémentaire for trek in Nepal: one and two (2010)
- Libretto for De la terreur des hommes (2011)
- Introduction to [Sic] by Davis Schneiderman (2013)
- "Embêtant" in Balafre (2014)
- Introduction to Sphinx by Anne Garréta, tr. Emma Ramadan (2015)
- Aphorisms in Twelve Aphorisms (2017)
- What Rhymes with Trick Dog? (2017)
- "Ways and Means," introduction to Collected Poems 1946–2016 by Harry Mathews (2020)
- "Equitable Nocturne" in Paris Atlantic 38 (2020)
- "often pastoral often breathing," introduction to Sexual Stealing by Wendy Walker (2021)
- Dear McSweeney's: Twenty-Two Years of Letters from Writers,
Readers, and the Occasional Bewildered Consumer (as editor, 2021)