Data Over Sound

Bootstrapping communications between two devices can be a challenge. While yourlong-term communications might be over WiFi, for example, you may have an initialstep of getting a device onto that WiFi network. This is reasonably common in IoTscenarios, where you have some device that has limited input options and you needto teach it a WiFi network SSID and passphrase, so it can try to connect.

There are many options for that bootstrapping process, though they all have theirhardware requriements:

Bluetooth or BLE NFC QR code scanned via a camera And so on

A lesser-used technique ��� at least in terms of obvious usage ��� is ���data over sound���.One device plays an audio clip that contains embedded data, and the other devicereceives it via a microphone and decodes it. Google Nearby uses this, in the formof near ultrasound, as one of its communications paths.

Google Nearby is nice but is closed source as part of Play Services. Google offersvery limited cross-platform support, and the system is fairly tightly tied to theGoogle Play ecosystem. This makes it impractical for many IoT scenarios, and italso limits the auditabilty of the code (from a privacy standpoint).

There have been various attempts at addressing this over the years. I had contemplatedcreating one (code name: chirpr) years ago and never found the time.

Georgi Gerganov is taking another stab at it, with an interesting approach. Hisfocus is purely on converting data to and from raw PCM audio clips, leaving it upto individual platforms to handle the actual audio I/O. For the data-to-sound conversions, hehas an MIT-licensed C/C++/WASM library thatallows for use on just about every platform imaginable. Right now, he has demosfor Android, iOS, and macOS/Linux/Windows desktops, in addition to the browser.He also has CLIs and a Web service for the raw sound clip conversions. Basically,he has proven that his code can run pretty much anywhere.

His Android and iOS demos though are just that: demos. What the project lacksis an Android library and iOS framework/Cocoapod for adding his data-over-soundcode to an app. I took a shot at it, but while I am pretty decent at Android overall,I am not strong with the NDK or with lower-level audio APIs. The Android librarywould need both of those in time, and so I am ill-suited to create the library,though I could help test and perhaps maintain it.

With luck, somebody else stronger in those areas will have a similar itch to scratchand will take the lead on creating a proper Android library for this project.