The Vecosek Ecosystem

I'm Seb Mondet:

• By Day: Software Engineering / Dev-Ops at Mount Sinai Health System, NYC
  • Previously within the Hammer Lab.
  • The NYC branch closed in Dec 2017.
  • Now working at the institute level.
• By Nights/Week-ends: Musician
  • {Bass,Electric,Classical} Guitar.
  • Various bands over the years.
  • One Electro/funk/drum'n'bass project.

• New-ish domain for most people.
• Example → Imagine you can generalize.
• A few OCaml libraries that work together:
  • Communicate through JSON (from atdgen).
  • Attempt at being very generic.
  • ISC || MIT licenses.
  • 80% released in Opam.

What's the need?

• Live music, partially improvised.
• Electronic music and human musicians.
• MIDI-based hardware.
• Flexibility.
• Multiscale Composition.

This is not:

• “Live coding.”
• Algorithmic music.

We want to …

• Trigger samples.
• “Play” virtual instruments (drums, synthetizers, etc.).
• Write/design composition with enough flexibility.
• Control the system in real-time with arbitrary input interfaces.

• Communication Protocol
  • Encoding of messages.
  • Hardware interoperability.
  • Early 80's …
• MIDI FIles? a.k.a. “SMF” → Extension to the protocol.
• General MIDI → Extension assigning a few default semantics to messages (1991).
• “One day, Open Sound Control will save us from 7-bit integers …”

Why on earth spend time writing software instead of just playing music …

• Market Leader: Ableton Live.
  • Big GUI-based editing/configuration.
  • Closed-source ⊕ close-data-formats.
• Seq24 (and forks Seq32, Sequence64 …)
  • Designed for live.
  • One level of abstraction, cf. Non-sequencer.
• JISS: LUA-based DSL (2012).
• …

Keyword: Flexibility

• No git.
• Different “mixes” from same lower-level music.
  • 45 minutes Vs One-hour gig.
  • Click / no-click.
  • Different input devices.
• How to think about music depends on the kind of music.
  • Intervals Vs Notes.
  • Repetitions / Variations.

Music on GNU/Linux → JACK .

Locoseq, 2007/2008.

• Previous geological era: Camlp4, GODI, Google Code hosting.
• Multi-scale MIDI compositions (notion of “meta-tracks”).
• LablGTK(2) Editor → Abandon ☹.
• “Archived” on Github: smondet/locoseq

Ten (10!) years later: code relatively easy to understand/reuse.

Too complex for graphical apps → EDSLs

• Vecosek: A headless JACK-MIDI sequencer.
  • Interprets a simple low-level “language.”
  • Focus on real-time / correctness.
  • Separate C thread → communication with JACK.
• Vimebac: A generic MIDI-driven Display/GUI.
  • Arbitrary text “instructions.”
  • Visual Metronome.
  • Custom Keyboard bindings: MIDI event source.

The vecosek-scene library, a.k.a. the very low-level “EDSL:”

• Events:
  • Track {starts,ends}.
  • Midi input (matching).
• Tracks: Sequences of relative-time × action.
• Actions:
  • Output MIDI Event.
  • Change BPM.
  • Start/stop track.
  • Add event handler.
  • …

Single Window Customizable GUI

• Wall with OpenGL + SDL (cf. @let-def 's talk ☺).
• Arbitrary Keyboard bindings → MIDI events.
• Option to run JACK-free → just visual metronome.
• Blocks screen-{lock,saver} …
• Protocol within MIDI, incl. text.

• Put together long (≥ 1 hour) interactive sessions.
• Ability to maintain multiple versions / orders: a.k.a. “Mixes”.
• Pre-configured limited interaction (MIDI-pedal-like).

Get MIDI Part

let module Tracks = Make_midi_track_finder (struct
  let path = "midi-tracks/break-a.mid"
end) in
let chord =
  Tracks.get_midi_part ~instrument:(`Named "bass-synth") "BreakA-chord-1" in
let melody = 
  Tracks.get_midi_part ~instrument:(`Named "melody") "BreakA-melody-1" in
(* ... *)

Tempo Ramp

let tempos =
  make_tempo_list ~start:176 ~stop:to_bpm ~length:14 @ [to_bpm; to_bpm]
in
List.mapi tempos (fun idx bpm ->
    let drum_choice =
      if 0 <= idx && idx < 8 then 1 else if idx < 12 then 2 else 3 in
    let msg =
      sprintf "BF slow down (%d BPM, [%d/%d], D%d)" bpm (idx + 1)
        (List.length tempos) drum_choice in
    let next, with_break =
      if idx + 1 = List.length tempos then ("BASS TONE", `Default)
      else ("slow down", `None) in
    let drum_part =
      bar_display_part 2 ++ multiply 2 (base drum_choice)
      ++ if idx mod 4 = 3 then concat [empty_44; brk drum_choice] else empty_44
    in
    list [tempo bpm; one ~next msg drum_part] )

Relative Notes

let main_melody =
  part ~length:Q.(i 8) "some weird melody"
    begin
      melody ~from:(a_0 + 12) [
        n 0 Q.(i 1 + 1/2); n (-2) Q.(1/2);
        n (-1) Q.(1/2); n 6 Q.(1/2);
        n 6 Q.(1/2); n 5 Q.(1/2);
        r Q.(i 1);
        n 2 Q.(1/2);
        n 1 Q.(1/2);
        n (-1) Q.(1/2);
        n (-2) Q.(2/2);
      ]
    end

Q Rational-ish Numbers → precise/consistent MIDI “Ticks”

module Q: sig
  type t = Fraction of int * int | Plus of t * t | Times of int * t | Minus of t
  val q : int -> int -> t
  val ( / ) : int -> int -> t
  val i : int -> t
  val one : t
  val ( + ) : t -> t -> t
  val ( * ) : int -> t -> t
  val minus : t -> t
  val ( - ) : t -> t -> t
  val half : t
  val third : t
  val quarter : t
  val to_float : t -> float
  val to_ticks : ppqn:int -> t -> int
  (* ... *)
end

High-Level Calls

let first_part =
  [ drum_test_pre_intro gp
  ; intro_and_blind_future gp
  ; the_pun gp
  ; spicy_thing ~with_solo:false ~ann_next:"Old Book" gp
  ; old_book ~with_end:`Alt ~ann_next:"BASS TONE" gp
  ; tempo 65
  ; bass_tone ~bpm:65 gp
  ; gently_reminded ~initial_bpm:65 gp
  (* ... *)

With duration estimations:

Total length: `01:02.62822`
- **No Moment**
- `00:00.000`: **Video Demo 0.1** (200 bpm) → `00:02.2399`
    - ∞ Hello (length: ((1 + 8) + (- 1))) (next: Some 4/4 riff)
- `00:02.2399`: **Video Demo: Master of Puppies** (200 bpm) → `00:19.19184`
    - ∞ Famous 4/4, master of pups, looping (length: (8 × 4)) (next: Famous 7/4)
- `00:21.21583`: **Video Demo: Money 7/4** (100 bpm) → `00:35.35938`
    - ∞ Some Drum'n'Bass (length: (4 × 4))
    - ∞ Famous 7/4 (length: 7) (next: Drum'n'Baaaaasss)

Improve documentation and:

• More interactivity (vi-like event sequences).
• Hot-reloading
• “Looper control” EDSL (WIP).
• Generate sheet music/tabs w/ high-level information.
• Control more things: lights, videos, …

Any questions?

Cf. also:

• https://gitlab.com/smondet/vecosek
• https://seb.mondet.org
• https://ni3.dance

URL: https://wiki.haskell.org/Haskore

• “Haskore is a set of Haskell modules for creating, analysing and manipulating music. Music can be made audible through various back-ends.”
• Static compositions, outputs to CSound, MIDI, WAV …
• Many dependent/specialized projects.

URL: https://wiki.haskell.org/Hommage

• “HOMMAGE is the successor of the Haskell Sound Lib. It was developed 2005-2006 […]”
• Actual homepage https://dvde.biz/projects/haskellommage is 404 now.

URL: http://hackage.haskell.org/package/HarmTrace

• “Harmony Analysis and Retrieval of Music with Type-level Representations of Abstract Chords Entities”

URL: http://hackage.haskell.org/package/mezzo

• “Mezzo is a Haskell library and embedded domain-specific language for music description. Its novelty is in the fact that it can enforce various rules of music composition statically, that is, at compile-time. This effectively means that if you write "bad" music, your composition will not compile – think of it as a very strict spell-checker for music.”

URL: https://tidalcycles.org/

• “TidalCycles (or Tidal for short) is a language for live coding patterns. It allows you to make musical patterns with text, describing sequences and ways of transforming and combining them, exploring complex interactions between simple parts.”
• Gihub: https://github.com/tidalcycles