Let’s start by looking at the project itself. Early in March 2005, the folks at Zenph studios invited me to a nearby recital hall to hear the famous pianist Glenn Gould play solo piano on a 9-foot concert grand, live for an audience of about 30-40 people. But here’s the twist: the piece was played exactly as he played it on a 1955 recording, and Gould himself died over twenty years ago.
Was this a pragmatic seance in operation? No. Player piano rolls? No. All the team had to go on was a well-known but scratchy mono recording made in 1955, and here was a live piano replaying the concert note for note.
And just to prove how identical the performance was, our host, John Q. Walker (formerly of Ganymede, now president of Zenph) played a recording of the new performance in one speaker and switched back-and-forth to the original mono recording in the other speaker. The sound was identical, except for the scratches, clicks and pops on the mono recording.
From that old vinyl mono recording, the team at Zenph Studios analyzed the audio waveforms and extracted every nuance of the original performance: articulation, technique, pedals; everything needed to recreate the performance. After analysis, this information is then stored in a high-definition MIDI file (which is a quite a bit more sophisticated than the more familiar, and now dated, MIDI standard). Finally, this data is fed to a freshly-tuned and voiced souped-up Disklavier Pro piano (of which only several currently exist in the world).
This is not the same Disklavier you might hear in a Marriott or Embassy Suites, endlessly banging out Misty for hapless passersby. This is a high-performance, cutting-edge instrument, capable of reproducing world-class, concert pianist performances.
Which brings us to the challenges faced by the Zenph team.
As with any bleeding-edge technology, the Disklavier Pro came with no technical specifications describing the high-definition MIDI format it uses to drive the instrument. As Zenph soon discovered, there were gaps in the data as well. Needless to say, there were no supporting libraries to deal with high-def MIDI files, either.
Then there’s the audio side of it. How do you determine which notes were hit, with what finger, articulation, the position of the piano pedals and all the rest of that when it’s mixed up in a monophonic audio track with all the ambiance, echo, and crowd noise? How on earth would you even unit test this sort of thing?
Oh, and modern startup companies, unlike the Foosball-equipped wastrels of the dot-com era, can’t afford to just throw money at a problem, so you need to adopt a very pragmatic approach.
We’ll see what they did in the next article.