Software

Share on Facebook
Tweet this
Share on LinkedIn
Share via E-mail

Software

  1. General Information
  2. Download Links
  3. Installing the App on Raspberry PI
  4. Installing the App on a Windows 10 PC
  5. Calibrating and Running the App
  6. Evaluation vs. Full Mode
  7. Configuration Parameters
  8. Debug PDF Files
  9. Troubleshooting Tips
  10. Extra Features: Voice Commands & Random Scrambling

1. General Information

The robot is driven by a Universal Windows Platform (UWP) application of our own creation called RubiksCubeRobot. By sending control signals to the robot's 8 servos and webcam, the application photographs the cube, performs the image and color recognition on the photos, determines the initial position of the cube, computes the sequence of rotations necessary to solve the cube, and then executes the sequence. The app can be used on Raspberry PI running Windows 10 IoT Core, or a regular PC running Windows 10.

In the evaluation mode, the app does not require a registration key, but is limited in functionality (see below for details). A permanent registration key is available for a small one-off fee. The license fees, which include life-time upgrades and maintenance, are as follows:

Personal use, discounted price: *$40.00
Personal use, full price:$50.00
Educational use:$75.00
Commercial use:$200.00

* The $10 discount for the personal-use license can be claimed if you post a photo of your robot on Thingiverse under "Post a Make".


2. Download Links

Please download the Raspberry PI, x64 and x86 versions of the app from the links below:

Download Link for Raspberry PI
Filename:RCR_3.0.107.0_ARM.zip
Version:3.0.107.0
Size:17.4 MB
Last Updated:2019-02-28
Platform:ARM (Raspberry PI)

Download Link for x64
Filename:RCR_3.0.107.0_x64.zip
Version:3.0.107.0
Size:17.5 MB
Last Updated:2019-02-28
Platform:x64

Download Link for x86
Filename:RCR_3.0.107.0_x86.zip
Version:3.0.107.0
Size:17.4 MB
Last Updated:2019-02-28
Platform:x86

3. Installing the App on Raspberry PI

The following instructions assume that you have already downloaded and installed Windows 10 IoT Core Dashboard on your local PC, installed the Windows 10 IoT Core operating system on your Class-10 Micro SD card, booted your Raspberry PI from it, and connected the PI to your local network via WiFi and/or an Ethernet cable. Your Raspberry PI device should be showing in the My devices list of the IoT Dashboard:

To install the RubiksCubeRobot onto your PI, please follow these easy steps:

  • Download the .zip archive for RubiksCubeRobot from the link above. Unzip it to a temporary directory of your PC's hard drive, such as c:\tmp.
  • Select Open in Device Portal from the IoT Dashboard. In the Windows Device Portal, go to Apps, Apps manager.
  • Under Install app, for App package, select the file with the extension .appx in the temporary directory, and for Certificate, select the file with the extension .cer.
  • Click on Add dependency three times. For the three Dependency boxes, select the three files in the \Dependencies\ARM subfolder of the temporary directory.
  • Click on Go under Deploy.

That's it! RubiksCubeRobot should now appear under Apps. You can start the application by choosing "Start" in the Actions drop-down box, and mark it as startup by clicking on the radio button in the Startup column.


4. Installing the App on a Windows 10 PC

Unzip the content of the download to a temporary directory such as "c:\temp". Prior to installing the app on your PC, you need to install the certificate in the Trusted Root Certification Authorities of both the Current User and Local Machine sections of the certificate store. This only needs to be done once.

Double-click on the .cer file in the temporary directory, click Install Certificate, select Current User, then select "Place all certificates in the following store", and select the Trusted Root Certificate Authorities folder. Repeat this procedure but this time select Local Machine instead of Current User.

Once the certificate is installed, double-click on the .appx file in the temporary directory to install the app on your PC.


5. Calibrating and Running the App

The main app screen looks as follows:

Photographing phase:

Assembling phase:

The blue and red buttons on the main page perform the following functions:

  • calibration -- takes you the Calibration Center where the servo target values are entered.
  • configuration -- takes you to the Configuration Center where the parameters responsible for image and color recognition can be viewed, and changed if necessary.
  • extras -- takes you to the Extra Configuration Parameters screen to enable/disable voice commands and random scrambling. These features are available as of version 3.0.107.0.
  • key -- allows you to enter and validate your registration key.

  • OPEN -- brings the rack-and-pinion servos to the full-back position and gripper servos to the neutral position so that the cube can be inserted.
  • RUN -- starts the work sequence after the cube has been inserted.
  • STOP -- performs an emergency stop.
  • OFF -- switches all servos off. If this button is pressed and held down for 3 seconds, the operating system shuts down (Raspberry PI only).
  • 📷 -- puts the grippers in the "full-forward" position, takes a picture, displays it, and then retracts the grippers. This button is useful for adjusting camera position and focus.

The 1st required step is to enter all the servo target values in the Calibration Center:

The screenshot above shows the settings for our robot, but you must obtain your own numbers during calibration using the Pololu Maestro Control Center software.

If your PC has more than one camera attached (laptops almost always have their own built-in camera), there is a 2nd required step: press the configuration button, and in the Configuration Center, select the robot's webcam via the Camera Name drop-down box. The other configuration parameters will be covered in detail in the next section. Go with the default parameters for your first test run.

Before clicking the RUN button, make sure all 8 servos are plugged into the Maestro servo controller, both the Maestro and webcam are plugged into the Raspberry PI's (or your PC's) USB ports, and the power source is connected to the Maestro.

Insert the cube and click the RUN button. Be prepared to hit STOP immediately if the cube slips out of the grippers, or some other malfunction occurs.

The robot will perform the necessary manipulations of the arms and grippers, and take 2 photographs for each face that are going to be displayed immediately. If image processing or color recognition fails, an error will be displayed, and the arms and grippers returned to the open position.


6. Evaluation vs. Full Mode

As of Version 3.0.98.0, the app can be evaluated without a registration key. In the evaluation mode, the solving sequence is limited to 5 steps. A 90°, 180° and 270° turn of a face is considered a single step. Therefore, without a key, the app is still capable of solving a slightly scrambled cube such as the one shown here:

The app displays the following message in the evaluation mode during the assembling phase:

Solving a thoroughly scrambled cube requires a registration key. Once the key is purchased, it needs to be entered in the app by pressing the KEY button. Enter your key and press Validate. Your device needs to be connected to the internet to complete the validation. Once the key is validated, the app no longer needs to be connected to the internet to function. Please contact us to purchase your permanent registration key, or send the correct licensing fee to our main email via PayPal.


7. Configuration Parameters

The app requires almost no configuring at all. The essential configuration parameters are: Debugging, Camera Name, Video Duration and Auto-Config Mode.

  • Debugging - If enabled, this toggle switch makes the app create a debug PDF file on the device, which is essentially an illustrated log file in which every step of the app's code is documented. Debug PDF files are covered in detail below.
  • Camera Name - Use this drop-down box to select the robot's camera if your device has multiple cameras attached to it. Laptops almost always do.
  • Video Duration - If set to a non-zero value, enables the delayed image capturing mode in which the camera shoots short videos and uses the last frame for analysis instead of taking static photos. Delayed image capturing gives the camera enough time to self-adjust to the surrounding lighting conditions and produce more vivid, easier to analyze images. By default, this value is 1000 ms (1 second). Setting this value to 0 (not recommended) makes the camera take static pictures instead of shooting short videos.
  • Auto-Config Mode - When auto configuration mode is enabled, the app performs color recognition based solely on the twelve photographs it has taken, without the need for any user-supplied color-related parameters. This mode is on by default.

The numerous hue, saturation, brightness and glare-related parameters are present for backwards compatibility with older versions of the app, but disabled (grayed out) by default. They are only used when the Auto-Config mode is switched off (not recommended.)

The Area Cutoff, Squareness, Angle Deviation and Size Deviation parameters control the square zone recognition code. The default values have been chosen by trial and error and are generally considered optimal. Changing them is not recommended.


8. Debug PDF Files

The app is designed to provide an insight into its thinking process. When the Debugging toggle switch is on, the app creates a PDF file for every run containing the photographs it has taken, color recognition results, and other information. On Raspberry PI, the PDF files can be found in the folder \Data\Users\DefaultAccount\Pictures. On a regular PC, this folder is This PC\Pictures.

A typical page from the PDF file looks as follows:

The top of the page displays the two photographs taken per cube's face with the zone detection results superimposed over the images. The center square is marked with a dashed outline, the other squares with a solid outline.

The color diagram on the middle left displays the color zones with their hue, saturation and brightness values. This information is generated by the old legacy code and discarded in the Auto-Config mode.

The color diagram on the middle right is the output of the color recognition module in the Auto-Config mode, before error correction is applied. Right below it, error correction log entries may be displayed (not shown here). At the bottom right, the final result, with error correction applied, is displayed.

The debug PDF files are a valuable troubleshooting tool. For example, during a run, the app has generated the error ERROR_UNSOLVABLE_CUBE. The PDF file generated during that run offers an instant explanation for the failure: a mechanical problem caused the cube to be way off center during photographing (see image below). As a result, the color zones were not correctly identified as half of the cube's face was not even within the camera's view.


9. Troubleshooting Tips

RUN Button is Clicked But Nothing Moves

Make sure your Maestro controller' serial mode is set to USB Dual Port via the Pololu Maestro Control Center (Serial Settings tab), and the Apply Settings button is clicked:

Camera not Centered

If the camera is not properly mounted or not pointing at the middle of the cube, a photo may look like the image above . If the app can't see the entire face of the cube, it can't process it. Make sure the center of the cube is roughly in the middle of the photograph. Use the Camera button to take test photos before a run. Adjust the camera's position in the camera holder if necessary. Also make sure the grippers hug the cube tightly to avoid shifting while the cube is being photographed. Adjust servo calibration values if necessary.

Images Appear Upside-Down

It has been reported by several users that the camera may take upside-down photographs. Apparently, in some of the cameras, the sensor is mounted upside-down. Upside-down images are guaranteed to cause the error ERROR_UNSOLVABLE_CUBE accompanied by a plethora of error correction entries in the debug PDF file:

If your camera has this problem, the entire camera assembly needs to be carefully unscrewed, removed from the frame, flipped upside-down and re-inserted into the frame.

Not Enough Light

If the photographs are too dark, some squares may appear completely black, like the square marked with a red arrow on the image below. This will cause the app to generate the error ERROR_COLOR_COULD_NOT_BE_DETERMINED. Make sure the cube is reasonably well lit, while avoiding glare.

Errors such as ERROR_TOOMANY_MISIDENTS, ERROR_SIDECUBIE_MISIDENT, etc.

The occurrence of an error containing the word MISIDENT indicates the auto-config mode is not enabled. Go to the Configuration center, make sure the Auto-Config Mode switch is on, and press Save even if you did not make any changes to the parameters.

The Colors are Identified Correctly but the Cube is Not Being Resolved

It is critical that your robot photograph the cube's faces in a particular order. Please refer to our introduction video for the correct sequence or rotations during photographing. For example, if you insert the cube with the white center square pointing towards the camera and red center square pointing upwards, then the correct order in which the center squares should appear on the photographs is:

White -> Red -> Yellow ->Orange -> Blue -> Green

If some or all of the gripper servos are calibrated incorrectly and turn from the neutral to 90° position in the wrong direction, the app won't be able to correctly reconstruct the cube's initial position and therefore, won't be able to resolve it.

Please do not hesitate to contact us if your robot keeps refusing to solve the cube. We will do our best to help you.


10. Extra Features: Voice Commands & Random Scrambling

As of Version 3.0.107.0, the app is equipped with a new button, Extras, which takes the user to the Extra Configuration Parameters screen. On this screen, two additional features can be enabled and disabled: Voice Commands and Scrambling. This section describes both features in detail.

10.1 Voice Commands

As of Version 3.0.107.0 of the app, the robot can be started and stopped via voice commands. To enable voice commands, press the EXTRAS button on the main screen, and put the Enable Voice Commands toggle switch in the On position on the Extra Configuration Parameters screen. Enter the phrases for the RUN and STOP commands. In our example, these phrases are "Solve this cube" and "Stop now", but you can pick your own commands.

The Confidence Threshold drop-down box specifies the level of confidence the voice recognition algorithm needs to have to execute a command vs. ignoring it. We recommend setting this option to "Low".

When the Enable Voice Commands option is enabled, an ear symbol is displayed in the status area of the main screen. When a command is received but not recognized, the ear symbol momentarily increases in size and goes back to its normal size.

When the app is running on a Windows 10 PC, the microphone needs to be enabled for the app. To enable it, go to Settings -> Privacy -> Microphone and make sure the toggle switch is on for the RubiksCubeRobot app, as follows:

If the microphone is not enabled, the ear symbol will be replaced by a yellow exclamation sign.

10.2 Random Scrambling

Following your numerous requests, we have equipped the robot with a random scrambling feature. To enable it, press the EXTRAS button on the main screen and put the Enable Scrambling toggle switch in the On position on the Extra Configuration Parameters screen. You also need to specify the number of commands, which should be greater than 0, and optionally a random generator seed value, which is an arbitrary text string.

When scrambling is enabled, the RUN button on the main screen becomes the SCRAMBLE button:

When the SCRAMBLE button is pressed, the robot generates a random sequence of turns, and then executes this sequence. The length of the sequence is specified via the Number of Commands parameter. A single command is a 90°, 180°, or 270° turn of the lower, upper, left, right, front or back face of the cube.

When the Seed Value parameter is empty, the sequence is completely random and unique for every invocation. When the Seed Value is non-empty, it is used as the seed value for the pseudo-random number generator employed by the app. Therefore, a certain seed value generates the same scrambling sequence for every invocation.


<< Hardware