Start Here: Basic Tutorial
No matter how you use Sound Control, you will always take the following steps, as shown in the video above:
- If this is your first time using Sound Control, make sure you have Sound Control installed and you have also done the necessary extra setup for any input devices that require it (i.e., Leap Motion, micro:bit).
- Plug in any external input devices you plan to use (e.g., Leap Motion, micro:bit, Gametrak, external camera or microphone), before launching the Sound Control software.
- Run Sound Control!
- Choose an input sensor to capture information about how the user/performer is moving.
- Choose a sound mode. Any input device can be used with any sound mode.
- “Train” the software by showing it what sounds to make when it sees different performer gestures/positions (see the video above to watch how to do this).
- Run the instrument to play it!
- Optionally, save your instrument to be able to play it again later, or change your instrument by changing how it is trained.
Detailed Setup Guidance
Installing Sound Control on Mac
Download the Mac version of the app from the “Downloads” page and follow the instructions below, or equivalently watch the following video.
- Double-click the .pkg file you have just downloaded.
- If you encounter an error message about an “unidentified developer,” simply go to System Preferences, select “Security & Privacy,” and select “Open Anyway” (you may have to enter your password). You will get a new dialogue box alerting you that the software is from an unidentified developer, but this time you will have the ability to click “Open.”
- Follow the installer dialogue to install the software. It will place the software in your Applications folder.
- Finally, navigate to your Applications folder and double-click on “Sound Control” to run the program.
- Note that you’ll also need to take a few extra steps to add support for Leap Motion or micro:bit if you wish to use these devices.
Installing Sound Control on Windows
Download the Windows version of the app from the “Downloads” page and follow the instructions below, or equivalently watch the following video.
- Right-click the .zip file you have just downloaded and select “Extract All.” For your destination, choose “C:/Program Files/” then click “Extract.”
- Navigate to C:/Program Files/ in Windows Explorer and double-click on “Sound Control” to run the program.
- Note that when you run Sound Control for the first time, you will need to grant it permission to send data over your network.
- Note that you’ll also need to take a few extra steps to add support for micro:bit if you wish to use it as an input device.
The Leap Motion is a USB device that gets information about hand position. If you are planning on using a Leap Motion with the app, first download the Leap SDK for Mac. Then watch the video below to learn how to install the Leap SDK software and to use the Leap Motion visualiser to check that Leap is working on your machine. Once you’ve verified Leap is working on your machine, you don’t ever need to run the visualiser again (though you can if you want). Simply make sure that the Leap is plugged in to your computer before launching the Sound Control software each time you want to use it.
The micro:bit is a small, inexpensive board that can be used as a tilt sensor with Sound Control.
Setting up micro:bit on Mac
Option 1 – To use the micro:bit while it is plugged in via USB on Mac:
- First, download the USB micro:bit file for Mac from http://soundcontrolsoftware.com/download/#microbit.
- Plug in the micro:bit to your computer via USB. As shown in the video below, this will cause your micro:bit to appear as an external drive on your computer, which you can see on your Desktop.
- Drag the MicroBit-USB file that you have just downloaded onto this micro:bit “drive”, as shown in the video below.
- Launch Sound Control and choose “Micro:Bit Position (USB)” from the “Pick Sensor” menu.
- Hit “refresh” next to “Select Port” and then select the port corresponding to your micro:bit from the dropdown menu. It is likely your micro:bit will show up looking something like “usbmodem” followed by a number. Once you have selected the right port, the X Y Z values (showing you the tilt of your micro:bit along three axes) will begin to change. This means your micro:bit is connected and ready to use! If you don’t see this happen after waiting a moment, select a different port from the dropdown menu.
Option 2 – To use micro:bit wirelessly over Bluetooth on a Mac:
- First, download the Bluetooth micro:bit file corresponding to your micro:bit version from http://soundcontrolsoftware.com/download/#microbit. If you don’t know which version you have, try using the new version (micro:bit 1.5); if this doesn’t work with Sound Control even after you follow all the instructions below, then try using the old version (1.0) instead. If all else fails, you can use the instructions above to use micro:bit plugged in via USB (see instructions above) instead of using it via Bluetooth.
- Plug in your micro:bit to your computer via USB. As shown in the video above, this will cause your micro:bit to appear as an external drive on your computer, which you can see on your desktop.
- Drag your downloaded micro:bit file (i.e., CBMicroBit-New or CBMicroBit-Old) onto your micro:bit “drive”, just as shown in the video above.
- To go fully wireless, unplug your micro:bit from your computer and plug it into an external micro:bit battery pack (make sure it’s charged and the micro:bit lights up!). Alternatively, if you don’t have a charged battery pack, you can keep the micro:bit plugged into a computer via USB so that it draws power from your USB port.
- Ensure your computer has Bluetooth turned on.
- Launch Sound Control and choose “Micro:Bit Position (Bluetooth)” from the “Pick Sensor” menu. The micro:bit should automatically connect to the application and show you updating X Y Z values, indicating that the tilt of the micro:bit is being received by Sound Control.
Setting up micro:bit on Windows (USB only)
- First, download the USB micro:bit file for Windows from http://soundcontrolsoftware.com/download/#microbit.
- Plug in the micro:bit to your computer via USB. As shown in the video below, this will cause your micro:bit to appear as an external drive on your computer, which you can see in a Windows Explorer window.
- Drag the MicroBit-USB file that you have just downloaded onto your micro:bit “drive” in a Windows Explorer window, just as you see in the video below.
- Launch Sound Control and choose “Micro:Bit Position (USB)” from the “Pick Sensor” menu.
- Hit “refresh” next to “Select Port” and then select the port corresponding to your micro:bit from the dropdown menu. It is likely your micro:bit will show up looking something like “COM” followed by a number. Once you have selected the right port, the X Y Z values (showing you the tilt of your micro:bit along three axes) will begin to change.
- If you have followed these instructions but do not get changing X Y Z values from micro:bit, you may have to download a driver to enable serial support for micro:bit on Windows. To do this, visit this website and follow the instructions to download the mbed Windows serial port driver and run the installer. Finally, unplug the micro:bit and plug it in again; then re-start Sound Control and repeat steps 4 and 5 above.
These example videos illustrate how to use the various sensors and sounds made available in the app. Remember any sensor can be combined with any sound to make a custom instrument for your needs.
About Each Input Device
Gametrak Real World Golf controller
The Gametrak “Real World Golf” controller enables a user to control the sound by changing the length and position of two strings attached to a plastic base. Unfortunately these controllers are no longer being manufactured, and most controllers being sold online at present do not work as USB devices, so we discourage anyone from buying one for the sole purpose of use with Sound Control.
This uses the webcam or an external USB camera with a very simple computer vision program. The “dumb” camera can enable Sound Control to respond to very dramatic changes in user position (e.g., leaning to one side or another, placing a hand in front of the camera) or background (e.g., pointing the camera at different places in a room). It is unlikely to work with very subtle changes (e.g., it probably can’t tell the difference between two people, or between two hand shapes).
This uses the webcam or an external USB camera to track the location of a coloured object in front of the camera, for instance playing one sound when the object is in the lower left, another sound when it’s in the lower right, and another sound when the object is absent. The “colour tracker & MIDI mapper” video below shows you how to choose which colour to track. You may wish to track a toy, a piece of clothing (e.g., a bandana tied to a head, or a bracelet on the hand), a piece of fruit, or something else. You’ll want to ensure that the colour you are tracking does not show up elsewhere in the field of view of the camera (e.g., ensure it’s not elsewhere on the user’s clothing, or in the background of the room).
Leap Motion (Mac only)
The Leap Motion is a specialised device that tracks the position of one or two hands in 3D space. Sound Control allows you to use Leap Motion to control sound with the palm position (e.g., playing one sound when the hand is on the left, another sound when the hand is on the upper right, another sound when the hand is absent) or with hand width (e.g., one sound when the fingers are closed tightly, another when they are spread wide). The Leap works best when its sensor is facing the palm of the hand, such as when it is placed on a tabletop under someone’s hand with their palm facing down. If you are interested in using a Leap Motion, we recommend checking on eBay and other second-hand sites for good deals. Please follow the instructions for installing Leap Motion before using it with Sound Control.
The BBC micro:bit is a small and affordable electronics device that is widely used in education. Sound Control enables you to use the tilt of the micro:bit in three axes to control sound. Note that micro:bit does not sense absolute position, such as whether it is to the left or to the right of your body; rather, it is good for things like holding it in your hand and sensing whether your hand is palm-up, palm-down, or halfway between. Additionally, micro:bit can be attached to clothing (e.g., bracelets, headbands, shoes), toys, or other objects to sense their tilt. Please follow the instructions for setting up micro:bit before using it with Sound Control.
Sound Control enables you to use the volume of sound coming into a microphone as a control signal. This is most effective when using an external microphone (e.g., a USB headset mic) with the microphone close to the user’s mouth and pointed away from the speaker you are using to play the Sound Control application’s own music. The intended use is for the user to speak, sing, or blow into the microphone in order to trigger sound from Sound Control, though you could use this in other ways.
Sound Control enables you to use the position of the mouse pointer on your computer screen to control the sound (e.g., playing one sound when the mouse is in the top left of the screen, and another sound when it is in the bottom right). You can use this with a built-in trackpad or an external mouse. Note that you will want to use Sound Control’s keyboard shortcuts to build your instrument when you’re using a mouse (pressing spacebar to record “Precise” examples and “Q” to record “Quick” examples). The “training” interface on the far right pane of Sound Control will remind you of these shortcut keys.
The Multi-colour tracker enables you to use up to three different coloured objects (sensed using a built-in or USB webcam) to drive changes in the sound. This input device does not respond to the location of the objects in front of the camera, but rather only their presence or absence. For instance, you may wish to play one sound when you hold up a red toy, another sound when you hold up a blue toy, another sound when you hold a yellow toy, and silence when you hold up none. Like the single-colour tracker, you’ll want to ensure you don’t try to track colours that also appear on the user’s clothing or in the background. The “multi-colour tracker & looper” example video shows you how to choose which colours you want to track.
About Each Sound Mode
The sample player allows you to trigger different pre-recorded audio files, one at a time. To get started, drag a folder containing at least a few audio files in MP3, WAV, or AIFF format onto the “Drop folder here” box, then use the “Select sample” drop-down menu to change which one is played (e.g., see the “Dumb camera and sample player” example below). You can choose to repeat each sample or to play it just once. You can download a variety of samples to get started at the Sound Control Downloads page.
The granulator allows you to change the speed, pitch, and volume of a repeating sound sample. You can choose among three built-in samples or use your own sample (WAV, MP3, or AIFF format). You can download a variety of samples to get started at the Sound Control Downloads page.
The looper allows you to turn up to six musical samples (“loops”) on and off, creating layered textures or soundscapes. Click on the pads (squares) numbered 1 through 6 to turn the sound loop corresponding to that pad on or off.
You can choose from three built-in sets of loops– reggae, ska, or garage style– or you can use your own samples. To use your own samples, choose “custom” on the left side of the looper window, then drag a different sound file onto each of the squares (“pads”) numbered 1 through 6.
You can download a variety of samples to get started at the Sound Control Downloads page.
The mixer allows you to adjust the volume (“mix”) of three sound samples, either built-in samples or your own. You can download a variety of samples to get started at the Sound Control Downloads page. To use your own samples, drag a different file onto each of the three volume sliders.
This sound mode uses FM synthesis to make wacky electronic sounds! You can adjust pitch, weirdness, and strangeness using the on-screen sliders, or hit “Randomise” to jump to a new random sound.
The MIDI mapper allows you to trigger different musical notes (and/or chords) within the app, or optionally through a third-party MIDI-enabled app such as Garage Band or Ableton Live. If you have a MIDI keyboard, you can also optionally connect this to Sound Control to make training your instrument more efficient.
To play within the Sound Control app:
This is the default mode of use. For each example, click on the keyboard on screen to turn notes on or off. You can select more than one note at a time to play a chord. Once you have your desired note(s) selected, put your input device where you want it and click “Precise.” Then deselect those notes and select new ones, and repeat. You can change the note length and velocity (volume) while you are training or while running; these must be changed manually and are not impacted by training.
To use a MIDI keyboard:
Plug in your keyboard before starting Sound Control. In the MIDI Mapper interface, hit “refresh” under “Optional external MIDI devices.” Choose your keyboard from the “In” dropdown box. You can now build your Sound Control instrument exactly as above, except that you can press keys on your MIDI keyboard instead of clicking on the on-screen keyboard.
To play with a third-party MIDI-enabled app:
If you are on a Windows machine, you will first need to download and install LoopMIDI to be able to route MIDI from Sound Control to another application. Do this before running Sound Control.
Once you are running Sound Control (on Windows or Mac), hit “refresh” under “Optional external MIDI devices” in the MIDI Mapper interface. Choose where you would like MIDI messages from Sound Control to be routed. Configure your third-party application to receive MIDI messages from this source. Your Sound Control application should now trigger sounds in your third-party application.