Let’s start by looking at the project itself. Early in March 2005,
the folks at Zenph studios invited me to a nearby recital hall to hear
the famous pianist Glenn Gould play solo piano on a 9-foot concert
grand, live for an audience of about 30-40 people. But here’s the
twist: the piece was played exactly as he played it on a 1955
recording, and Gould himself died over twenty years ago.
Was this a pragmatic seance in operation? No. Player piano rolls?
No. All the team had to go on was a well-known but scratchy mono
recording made in 1955, and here was a live piano replaying the
concert note for note.
And just to prove how identical the performance was, our host, John
Q. Walker (formerly of Ganymede, now president of Zenph) played a
recording of the new performance in one speaker and switched
back-and-forth to the original mono recording in the other speaker.
The sound was identical, except for the scratches, clicks and
pops on the mono recording.
From that old vinyl mono recording, the team at Zenph Studios analyzed
the audio waveforms and extracted every nuance of the original
performance: articulation, technique, pedals; everything needed to
recreate the performance. After analysis, this information is then
stored in a high-definition MIDI file (which is a quite a bit more
sophisticated than the more familiar, and now dated, MIDI
standard). Finally, this data is fed to a freshly-tuned and voiced
souped-up Disklavier Pro piano (of which only several currently exist
in the world).
This is not the same Disklavier you might hear in a Marriott or
Embassy Suites, endlessly banging out Misty for hapless
passersby. This is a high-performance, cutting-edge instrument,
capable of reproducing world-class, concert pianist performances.
Which brings us to the challenges faced by the Zenph team.
As with any bleeding-edge technology, the Disklavier Pro came with
no technical specifications describing the high-definition MIDI format
it uses to drive the instrument. As Zenph soon discovered, there were
gaps in the data as well. Needless to say, there were no supporting
libraries to deal with high-def MIDI files, either.
Then there’s the audio side of it. How do you determine which
notes were hit, with what finger, articulation, the position of the
piano pedals and all the rest of that when it’s mixed up in a
monophonic audio track with all the ambiance, echo, and crowd noise?
How on earth would you even unit test this sort of thing?
Oh, and modern startup companies, unlike the Foosball-equipped
wastrels of the dot-com era, can’t afford to just throw money at a
problem, so you need to adopt a very pragmatic approach.
We’ll see what they did in the next article.