The Vecosek Ecosystem

Call for Collaboration: Live MIDI Sequencing with OCaml EDSLs

Seb Mondet
FARM Workshop, Sat, 29 Sep 2018

Who is this already?

I'm Seb Mondet:

  • By Day: Software Engineering / Dev-Ops at Mount Sinai Health System, NYC
    • Bionformatics.
    • Dev-ops.
  • By Nights/Week-ends: Musician
    • {Bass,Electric,Classical} Guitar.
    • Various bands over the years.
    • One Electro/funk/drum'n'bass project.

This Talk is a Call For Contribution

  • Using example to run through ideas.
  • Attempt at being very generic.
  • A few OCaml libraries/applications:
    • Point of communication: JSON files.
    • → room for other languages
    • All ISC || MIT licenses.

Context

What's the need?

  • Live music, partially improvised.
  • Electronic music and human musicians.
  • MIDI-based hardware.
  • Flexibility.
  • Multiscale Composition.

This is not:

  • “Live coding.”
  • Algorithmic music.

Example Setup: Laptop

NI3 Dance on stage with laptop

Example Setup: MIDI Controller

NI3 Dance on stage with laptop

Requirements

We want to …

  • Trigger samples.
  • “Play” virtual instruments (drums, synthetizers, etc.).
  • Write/design composition with enough flexibility.
  • Control the system in real-time with arbitrary input interfaces.

The MIDI Standard

Lingua Franca of electronic Music
  • Communication Protocol
    • Encoding of messages.
    • Hardware interoperability.
    • Early 80's …
  • MIDI FIles? a.k.a. “SMF” → Extension to the protocol.
  • General MIDI → Extension assigning a few default semantics to messages (1991).
  • “One day, Open Sound Control will save us from 7-bit integers …”

Existing Soutions

Why on earth spend time writing software instead of just playing music …

  • Market Leader: Ableton Live.
    • Big GUI-based editing/configuration.
    • Closed-source ⊕ close-data-formats.
  • Seq24 (and forks Seq32, Sequence64 …)
    • Designed for live.
    • One level of abstraction, cf. Non-sequencer.
  • JISS: LUA-based DSL (2012).

Hacker Unfriendly

Keyword: Flexibility

  • No git.
  • Different “mixes” from same lower-level music.
    • 45 minutes Vs One-hour gig.
    • Click / no-click.
    • Different input devices.
  • How to think about music depends on the kind of music.
    • Intervals Vs Notes.
    • Repetitions / Variations.

Open-source Context

Music on GNU/Linux → JACK .

QJackCtl Screenshot
  • Pseudo-real-time samples-stream + MIDI-messages IPC.
  • Decent well-documented C API.
  • ALSA MIDI still there.

Example Setup

Overview of the Example

Example Setup, A Bit Later

Overview of the Example A Couple of Weeks Later

Gloves Off

Too complex for graphical apps → EDSLs

  • Vecosek: A headless JACK-MIDI sequencer.
    • Interprets a simple low-level “language.”
    • Focus on real-time / correctness.
    • Separate C thread → communication with JACK.
  • Vimebac: A generic MIDI-driven Display/GUI.
    • Arbitrary text “instructions.”
    • Visual Metronome.
    • Custom Keyboard bindings: MIDI event source.

Vecosek-scene

The vecosek-scene library, a.k.a. the very low-level “EDSL:”

  • Events:
    • Track {starts,ends}.
    • Midi input (matching).
  • Tracks: Sequences of relative-time × action.
  • Actions:
    • Output MIDI Event.
    • Change BPM.
    • Start/stop track.
    • Add event handler.

The Vimebac Interface

Single Window Customizable GUI

  • OpenGLES + SDL (@let-def's Wall library).
  • Arbitrary Keyboard bindings → MIDI events.
  • Option to run JACK-free → just visual metronome.
  • Blocks screen-{lock,saver} …
  • Protocol within MIDI, incl. text.

The Vimebac Interface

GIF of Vimebac's Default UI

Vimebac, In Action

GIF of NI3's Vimebac

Vecosek AMA1

“Arbitrary Music API 1”(WIP):
  • Put together long (≥ 1 hour) interactive sessions.
  • Ability to maintain multiple versions / orders: a.k.a. “Mixes”.
  • Pre-configured limited interaction (MIDI-pedal-like).

Vecosek-AMA1, Example 0

Get MIDI Part

Vecosek-AMA1, Example 1

Tempo Ramp

Vecosek-AMA1, Example 2

Relative Notes

Vecosek-AMA1, Example 3

Q Rational-ish Numbers → precise/consistent MIDI “Ticks”

Vecosek-AMA1, Example 4

High-Level Calls

Vecosek-AMA1, Markdown Summary

With duration estimations:


Total length: `01:02.62822`
- **No Moment**
- `00:00.000`: **Video Demo 0.1** (200 bpm) → `00:02.2399`
    - ∞ Hello (length: ((1 + 8) + (- 1))) (next: Some 4/4 riff)
- `00:02.2399`: **Video Demo: Master of Puppies** (200 bpm) → `00:19.19184`
    - ∞ Famous 4/4, master of pups, looping (length: (8 × 4)) (next: Famous 7/4)
- `00:21.21583`: **Video Demo: Money 7/4** (100 bpm) → `00:35.35938`
    - ∞ Some Drum'n'Bass (length: (4 × 4))
    - ∞ Famous 7/4 (length: 7) (next: Drum'n'Baaaaasss)

Dependency Graph

The dependencies between packages

Future Work

Improve documentation and:

  • More interactivity (vi-like event sequences).
  • Hot-reloading
  • “Looper control” EDSL (WIP).
  • Generate sheet music/tabs w/ high-level information.
  • Control more things: lights, videos, …

The End

Any questions?

Ask NOW!

Cf. also:

Haskore

URL: https://wiki.haskell.org/Haskore

  • “Haskore is a set of Haskell modules for creating, analysing and manipulating music. Music can be made audible through various back-ends.”
  • Static compositions, outputs to CSound, MIDI, WAV …
  • Many dependent/specialized projects.

Homage

URL: https://wiki.haskell.org/Hommage

HarmTrace

URL: http://hackage.haskell.org/package/HarmTrace

  • “Harmony Analysis and Retrieval of Music with Type-level Representations of Abstract Chords Entities”

Mezzo

URL: http://hackage.haskell.org/package/mezzo

  • “Mezzo is a Haskell library and embedded domain-specific language for music description. Its novelty is in the fact that it can enforce various rules of music composition statically, that is, at compile-time. This effectively means that if you write "bad" music, your composition will not compile – think of it as a very strict spell-checker for music.”

Tidal

URL: https://tidalcycles.org/

  • “TidalCycles (or Tidal for short) is a language for live coding patterns. It allows you to make musical patterns with text, describing sequences and ways of transforming and combining them, exploring complex interactions between simple parts.”
  • Gihub: https://github.com/tidalcycles