Set and Get Sources for p5.Input()

[ogbabydiesal]

The original library had a getSources() and setSources() method for p5.Input(), new p5.sound should have this as well.

[atomiczagnut]

Hey, I am a new contributer here, and thinking of applying for GSoC.

I have used Processing in the past, and would love to contribute.

I think this could be a great first issue.

Thanks!

[ogbabydiesal]

excellent ok:

you probably saw these but take a look at the original library for an example pattern (it doesn't necessarily need to follow this if you think something makes more sense):
https://p5js.org/reference/p5.AudioIn/getSources/
https://p5js.org/reference/p5.AudioIn/setSource/

it would be nice to stick with inheriting Tone.js approach under the hood if that makes sense as well
https://tonejs.github.io/docs/15.1.22/classes/UserMedia.html#deviceId

[NiharikaHari]

Hey @ogbabydiesal, since I don't see any updates on this issue for a while, I'd love to contribute here.

Here's my plan -

• Implementation in src/sources/AudioIn.js.
• For getSources, will use UserMedia.enumerateDevices() to return the list of audio input devices. Query here - Do we want to include all fields (deviceId, groupId, kind, label) when returning the list?
• For setSource, will add deviceId to the constructor, update that in setSource, and then close the existing steam and open it with the passed device. Question - Since UserMedia.open() supports device ID or label or index, do we want the user to input only index like it was in the original library, or only device ID, or any of ID/label/index?

Please let me know if I can work on this, and if I should do anything differently. Thanks!

[ogbabydiesal]

Hi @NiharikaHari sorry i let this slip, if you are still available to work on it, I would ask, how does the original library handle those questions, I think we can follow a similar pattern but using p5 core libraries new async setup function where we would do anything in preload() before

[Abhishek9639]

Hey @ogbabydiesal @NiharikaHari, if this is still up for grabs I'd like to pick it up. Here's how I'd approach it, wanted to check with you both before opening a PR.

How the original library did it

Dug into p5.js-sound's audioin.js to answer the question you raised:

• getSources() wraps navigator.mediaDevices.enumerateDevices(), filters for kind === 'audioinput', and caches the resulting array on the instance (p5sound.inputSources). It returns a Promise and also supports a success/error callback pair for backward compatibility.
• setSource() does not take a deviceId directly. It takes a Number, the index into the array that getSources() returned, and looks up the actual device from the cached list internally. currentSource on the instance tracks that index.

So the original design deliberately kept setSource index-based rather than deviceId-based, probably to keep the API simple for beginners (setSource(0) reads nicer than passing a raw deviceId string).

Proposed implementation for p5.Input

Roughly matching that pattern but built on Tone.UserMedia:

• getSources()
  • Calls navigator.mediaDevices.enumerateDevices(), filters to audioinput.
  • Caches the filtered list on the instance (e.g. this._sources).
  • Returns a Promise resolving to that array (drop the callback style since the rest of the new library is Promise/async-first).
• setSource(num)
  • Keep it index-based like the original for parity, num is a position in the array from getSources().
  • Internally resolve this._sources[num].deviceId, close the existing Tone.UserMedia stream, and reopen with open(deviceId) (Tone's UserMedia.open() accepts a deviceId or label string directly, so this maps cleanly).
  • Update this.currentSource to the new index for parity with the old currentSource property.

One thing worth flagging: enumerateDevices() only returns proper label and full deviceId values after mic permission has been granted, before that device labels come back blank. So getSources() is only really useful after .start() (or some other call that triggers getUserMedia) has run at least once. Might be worth a note in the docs, or we return the sources anyway with a console warning if permission hasn't been granted yet, so the sketch doesn't just silently fail.

On the preload() point

Since getSources() already returns a Promise, it should be async-preload-compatible out of the box if we treat it like other async source-loading methods in p5, no extra plumbing needed on our end beyond making sure it doesn't require a callback to resolve. I wouldn't force it to run inside preload() though, since that would mean every sketch using p5.Input prompts for mic permission before setup() even runs, which feels like a bad default. I'd keep it callable from setup() and just make sure it works fine if someone does call it from preload().

Let me know if this lines up with what you had in mind, happy to adjust the setSource signature to accept a deviceId string as well if you'd rather move away from index-based lookups. Will open a PR once you're good with the direction.

[ogbabydiesal]

hi @Abhishek9639 sorry for the delay, this sounds good, keeping it in sync with the original library would be awesome (with the exception of now using async function instead of preload), is this something you can still work on?

[Abhishek9639]

Yes, absolutely! I’d be happy to work on this. I’ll keep it aligned with the original library while using an async function instead of preload.