Arduino Firmata Actor

liminal_andy

It would be an interesting plugin to have an official Isadora firmware to flash onto the Arduino that is essentially paired with a user actor that maps all TX pins to actor inputs and all RX pins to actor outputs. This would allow you to write voltages to pins directly from Isadora and read them back into Isadora as well, with all the communications handled for you to help out our non-programmer friends. This would limit the Arduino but would be a fast quickstart for those who just want to get data to and from their Isadora patch, and it should not require anything special as I think it can all be done with the existing serial communications tools.

Does this already exist? If not, could be a nice weekend project for me, if I ever reclaim a weekend!

mark

@liminal_andy said:

t would be an interesting plugin to have an official Isadora firmware to flash onto the Arduino that is essentially paired with a user actor that maps all TX pins to actor inputs and all RX pins to actor outputs.

My response here relates also to what @bonemap said above.

The problem with this plan is that many pins on the board can function as inputs or as outputs. Furthermore, how can we know if pin 5 is connected to an LED for which requires an on or off digital output, or to a servo motor output which requires a 14 bit value? The physical hardware connections would determine how you want the pins to function, and -- unless I'm missing something -- there is no way for Isadora (or any other software) to sense this hardware design.

[EDIT: we now have our own version of the Firmata Test App that is compatible with Big Sur. If you're on any version of macOS, you should be using this version. The latest version can always be found in the releases section of Firmata Test App on GitHub]

Downloading and trying the Firmata Test App might be informative. You'll see that you, as the user, need to choose what the function and direction of each pin is. It is not automatic.

The only way to implement Andy's plan is to force certain pins to be dedicated to certain modes and say to the user "pin 2 is digital input, pin 3 is digital output, pin 5 is for pulse width modulation output", etc. But wouldn't this be too limiting?

That said, the pin specification I've proposed above above could do exactly that by offering a pre-defined list of pin specifications as a default.

Thoughts?

Best Wishes,
Mark

liminal_andy

@mark Totally understand your point. We would be limiting the hardware dramatically by making these choices for the user. My thinking is that such a choice would allow me to hand a board to someone and say, "Here's the board, here's the actor, let's play." An early user may not need all of this functionality, they just need some PWM pins and some sense capabilities. When they are ready to grow out of the presets we created, they at least have the foundation. It's training wheels, but I feel strongly about getting more artists ready to use low level hardware because of the implications of physically reactive design.

mark

@liminal_andy said:

We would be limiting the hardware dramatically by making these choices for the user. My thinking is that such a choice would allow me to hand a board to someone and say, "Here's the board, here's the actor, let's play." An early user may not need all of this functionality, they just need some PWM pins and some sense capabilities.

Alright, I'm hearing you. Duly noted.

Best Wishes,
Mark

bonemap

@mark

Hi,

But the Firmata implementation does not replace the Serial interface group of actors -Serial In Watcher, Send Serial etc. does it? So in the overall scope of Isadora’s capability to interface with Arduino a simplified approach with Firmata will have no loss of extended functionality. As has been commented by a number of responses in this thread - the implementation is aimed at non-programmers and students/beginners. Some loss of extended functionality/limitation as a compromise towards simplicity and ease of use or as a quick prototyping, or introductory step to using a microcontroller with Isadora, is worth considering. Even as a non-programmer I know that I can turn to/take the next step towards serial functionality if I ever reach that level of sophistication with interfacing microcontrollers with Isadora.

So I think there is some merit in reflecting on a direct and albeit limited approach for the sake of plug and play simplicity.

Best wishes

Russell

mark

@bonemap said:

But the Firmata implementation does not replace the Serial interface group of actors -Serial In Watcher, Send Serial etc. does it?

No it does not replace them, it is an addition to the existing serial actors. This new actor will handle all the serial input and output internally.

We are always (painfully) mindful of backwards compatibility. We'd never take away an actor as fundamental as that.

Best Wishes,
Mark

bonemap

@mark said:

We'd never take away an actor as fundamental as that.

And I would say there is good, workable documentation and instruction for using Arduino with the Serial In Watcher, Send Serial Data actors. It is available through Isadora knowledge base. At least enough for a non-programmer like myself to succeed.

Best wishes

Russell

fubbi

What are the advantages of serial communication for the opportunistic prototyper?

I believe the future of accessible micro-controlling is ESP32 (web server, bluetooth low energy, processing power) and Teensy (Out of the box HID or Midi interface).

mark

@fubbi said:

What are the advantages of serial communication for the opportunistic prototyper?

That's a separate discussion, which you can most certainly start a thread about. Let's stick to the question at hand, OK? (Most Arduinos come with a USB serial interface connector that appears as a serial connection, as you know.)

Best Wishes,
Mar

mark

@bonemap said:

And I would say there is good, workable documentation and instruction for using Arduino with the Serial In Watcher, Send Serial Data actors. It is available through Isadora knowledge base. At least enough for a non-programmer like myself to succeed.

I'm mostly going on the multiple requests I've gotten on this over the years. On the Facebook post I made about this, several educators were immediately enthusiastic, so I would see merit in finishing this up. (It was basically two days of time while I waited for the team to test other stuff.)

Best Wishes,
Mark

bonemap

@mark

Super happy to get hands on with the Arduino Firmata implementation - if it is ready let’s try it.

Best wishes

Russell

tonyschuite

@fubbi

by knowing about ESP32 you already know about the Arduino more than the focus group for this actor and I would recommend just using the serial actors.

@marc

Reading the comments here, I think there is some good insights. as to the practical question of how you would implement an auto populated actor: after populating you would set the ins and outs to "none". If you then set one of the pins to be output, you disable the input field and vice versa, so it cannot be modified anymore.

Also, in regards to it being too many options: there are a fair few actors that have this. I don't think it is that much of a worry. however, you could go the route of only show the first 4 pins and add in the "help" section that you can enable more capabilities.

mark

@tonyschuite @bonemap @liminal_andy @juriaan

So, I've listened to all you have to say, and made substantial changes to the actor. I've explained how it's working now in this video, as it was easier to do this than to write it all out.

Please let me know if you think will help make this accessible to beginners and students who might use this actor.

Best Wishes,
Mark

Juriaan

@mark

Chapeau. That looks amazing and simple enough for students/people that never touched Arduino before. Just tell the system what you want, and be done with it

I could not wish for anything more, I love that it also labels the names of the Input / Output of the actor with the Pin / Type of pin that you generated. PWM, Analog, Digital, Encoder and Servo.

bonemap

@mark said:

I've explained how it's working now in this video

That appears very straight forward! nice solution with the innovation of the button to generate/automate connection ports, which I don't recall being apart of any other Isadora actor. Is it just Arduino's or are other boards suitable?

So the process would be grab the Firmata Standard Arduino Code and load it onto whatever flavour of board is at hand - using the Arduino IDE or similar. Connect the board through USB to the Serial Setup in Isadora as usual. Run the Firmata actor and check for a connection by observing the 'firmware' output. Determine what input and output ports are required for the project, enter these parameters and click the generate button - start linking data throughput with the rest of the patch.

Great !

What happens if I decide I need more ports after I have generated the initial ones can I go back and add through the same process or do I need to add through the edit text parser?

Best Wishes

Russell

mark

@bonemap said:

Are other boards suitable?

This actor relies the Firmata protocol, which allows things like asking the board what it's capabilities are (e.g. how many pins do you have? what can they do?) So if the board (whatever it is) runs Firmata, it would work.

So the process would be grab the Firmata Standard Arduino Code and load it onto whatever flavour of board is at hand - using the Arduino IDE or similar. Connect the board through USB to the Serial Setup in Isadora as usual. Run the Firmata actor and check for a connection by observing the 'firmware' output. Determine what input and output ports are required for the project, enter these parameters and click the generate button - start linking data throughput with the rest of the patch.

Yes, that's it.

What happens if I decide I need more ports after I have generated the initial ones can I go back and add through the same process or do I need to add through the edit text parser?

Well, right now it would overwrite whatever's there. If you've not modified it by hand, no biggie. But otherwise you'd lose your changes.

But if @tonyschuite agrees that this would make it easy for the students he teachers, I'll get this ready for distribution so people can try it.

Best Wishes,
Mark

jfg

@mark

Great and very useful. A lot of students will love it.

best

Jean-François

DusX

@mark this is awesome!!! Really fantastic!!

bonemap

@mark

Hi,

The Actor is amazing as is but Some additional reflecting on the procedure outlined in your video. There is an element of guess work in letting the actor report back on the availability of ports. I think I would want to know what a board is capable of before making the setting port parameters and generating them. If the board rejected a parameter setting I would want to know why.

A little online research into the specification of a particular board would probably be all that is required. Another approach might be to consider having an option to reveal the full capacity of the board through the actor- so there is some clear guideline to what the board is capable of before setting port assignments. Could the process be, using the existing actor as proposed, to trigger settings with an aim to reveal all of the the boards potential ports and then pair back/assign the ports on a second pass?

Or perhaps a dialog is populated with the boards port type and capacity when I select the port number... then I would be equiped with the information required to make appropriate port assignments for the board..

The demonstrated actor is great as is - so these are just additional thoughts.

Best wishes

Russell

Armando

@mark Amazing work Mark !

The only thing I don't understand in the video is why outputs are on the right in the actor and inputs on the left....

Ciao !