Software

Share on Facebook
Tweet this
Share on LinkedIn
Share via E-mail

Software

  1. General Information
  2. Download Links & Release Notes
  3. Installing the App on Raspberry PI
  4. Installing the App on a Windows 10 PC
  5. Calibrating and Running the App
  6. Evaluation vs. Full Mode
  7. Configuration Parameters
  8. Debug PDF Files
  9. Troubleshooting Tips
  10. Extra Features: Voice Commands & Random Scrambling

1. General Information

The robot is driven by a Universal Windows Platform (UWP) application of our own creation called RubiksCubeRobot. By sending control signals to the robot's 8 servos and webcam, the application photographs the cube, performs the image and color recognition on the photos, determines the initial position of the cube, computes the sequence of rotations necessary to solve the cube, and then executes the sequence. The app can be used on Raspberry PI running Windows 10 IoT Core, or a regular PC running Windows 10.

In the evaluation mode, the app does not require a registration key, but is limited in functionality (see below for details). A permanent registration key is available for a small one-off fee. The license fees, which include life-time upgrades and maintenance, are as follows:

Personal use, discounted price: *$40.00
Personal use, full price:$50.00
Educational use:$75.00
Commercial use:$200.00

* The discounted price for the personal-use license can be claimed if you post a photo of your robot on Thingiverse under "Post a Make".


2. Download Links & Release Notes

Please download the Raspberry PI, x64 and x86 versions of the app from the links below:

Download Link for Raspberry PI
Filename:RCR_3.0.114.0_ARM.zip
Version:3.0.114.0
Size:18.5 MB
Last Updated:2020-09-06
Platform:ARM (Raspberry PI)

Download Link for x64
Filename:RCR_3.0.114.0_x64.zip
Version:3.0.114.0
Size:18.6 MB
Last Updated:2020-09-06
Platform:x64

Download Link for x86
Filename:RCR_3.0.114.0_x86.zip
Version:3.0.114.0
Size:18.4 MB
Last Updated:2020-09-06
Platform:x86

What's New in Version 3.0.114.0:

  • 1280x720 and 1920x1080 support. The new version of the app supports high-resolution cameras that have recently become very common. Previous versions only supported cameras producing 640x480 photos, while the new version also supports 1280x720 and 1920x1080 cameras.
  • Support for upside-down photos. If your camera takes upside-down photos, you would previously need to flip over your entire camera assembly. With the new version, just turn on the Rotate Photos 180° toggle switch on the Configuration screen.

  • The Auto-Config mode is now permanently on and cannot be switched off.

3. Installing the App on Raspberry PI

The following instructions assume that you have already downloaded and installed Windows 10 IoT Core Dashboard on your local PC, installed the Windows 10 IoT Core operating system on your Class-10 Micro SD card, booted your Raspberry PI from it, and connected the PI to your local network via WiFi and/or an Ethernet cable. Your Raspberry PI device should be showing in the My devices list of the IoT Dashboard:

To install the RubiksCubeRobot onto your PI, please follow these easy steps:

  • Download the .zip archive for RubiksCubeRobot from the link above (the ARM version). Unzip it to a temporary directory of your PC's hard drive, such as c:\tmp.
  • Select Open in Device Portal from the IoT Dashboard. In the Windows Device Portal, go to Apps, Apps manager.
  • Click Choose File and select the file with the extension .appx from the temporary directory. Check the box Allow me to selection optional packages. Click Next.
  • On the next screen, select all three files in the /Dependencies/ARM subdirectory of the temporary directory, one by one. Click Install.

That's it! rcr3d (the name of the app) should now appear under Apps. You can start the application by choosing "Start" in the Actions drop-down box, and mark it as startup by clicking on the radio button in the Startup column.

WARNING: Not every camera works correctly on Raspberry PI. Some high-definition cameras produce distorted pictures and cannot be used.

4. Installing the App on a Windows 10 PC

Method 1 (recommended):

Follow this link to install the app directly from the Microsoft Store:

https://www.microsoft.com/store/apps/9NNCXHKWPGCZ

Method 2:

Unzip the content of the download to a temporary directory such as "c:\temp". Prior to installing the app on your PC, you need to install the certificate in the Trusted Root Certification Authorities of both the Current User and Local Machine sections of the certificate store. This only needs to be done once.

Double-click on the .cer file in the temporary directory, click Install Certificate, select Current User, then select "Place all certificates in the following store", and select the Trusted Root Certificate Authorities folder. Repeat this procedure but this time select Local Machine instead of Current User.

Once the certificate is installed, double-click on the .appx file in the temporary directory to install the app on your PC.


5. Calibrating and Running the App

The main app screen looks as follows:

Photographing phase:

Assembling phase:

The blue and red buttons on the main page perform the following functions:

  • calibration -- takes you the Calibration Center where the servo target values are entered.
  • configuration -- takes you to the Configuration Center where the parameters responsible for image and color recognition can be viewed, and changed if necessary.
  • extras -- takes you to the Extra Configuration Parameters screen to enable/disable voice commands and random scrambling. These features are available as of version 3.0.107.0.
  • key -- allows you to enter and validate your registration key.

  • OPEN -- brings the rack-and-pinion servos to the full-back position and gripper servos to the neutral position so that the cube can be inserted.
  • RUN -- starts the work sequence after the cube has been inserted.
  • STOP -- performs an emergency stop.
  • OFF -- switches all servos off. If this button is pressed and held down for 3 seconds, the operating system shuts down (Raspberry PI only).
  • 📷 -- puts the grippers in the "full-forward" position, takes a picture, displays it, and then retracts the grippers. The status area displays the camera resolution. This button is useful for adjusting camera position and focus. You can also use it to check if your camera is taking upside-down pictures.

The 1st required step is to enter all the servo target values in the Calibration Center:

The screenshot above shows the settings for our robot, but you must obtain your own numbers during calibration using the Pololu Maestro Control Center software.

If your PC has more than one camera attached (laptops almost always have their own built-in camera), there is a 2nd required step: press the configuration button, and in the Configuration Center, select the robot's webcam via the Camera Name drop-down box. The other configuration parameters will be covered in detail in the next section.

Before clicking the RUN button, make sure all 8 servos are plugged into the Maestro servo controller, both the Maestro and webcam are plugged into the Raspberry PI's (or your PC's) USB ports, and the power source is connected to the Maestro.

Insert the cube and click the RUN button. Be prepared to hit STOP immediately if the cube slips out of the grippers, or some other malfunction occurs.

The robot will perform the necessary manipulations of the arms and grippers, and take 2 photographs for each face that are going to be displayed immediately. If image processing or color recognition fails, an error will be displayed, and the arms and grippers returned to the open position.


6. Evaluation vs. Full Mode

The app can be evaluated without a registration key. In the evaluation mode, the solving sequence is limited to 5 steps. A 90°, 180° and 270° turn of a face is considered a single step. Therefore, without a key, the app is still capable of solving a slightly scrambled cube such as the one shown here:

The app displays the following message in the evaluation mode during the assembling phase:

Solving a thoroughly scrambled cube requires a registration key. Once the key is purchased, it needs to be entered in the app by pressing the KEY button. Enter your key and press Validate. Your device needs to be connected to the internet to complete the validation. Once the key is validated, the app no longer needs to be connected to the internet to function. Please contact us to purchase your permanent registration key, or send the correct licensing fee to our main email via PayPal.


7. Configuration Parameters

The app requires almost no configuring at all. The essential configuration parameters are: Rotate Photos 180°, Debugging, Camera Name, and Video Duration.

  • Rotate Photos 180° - Some webcams available on the market take upside-down pictures. This blocks the app's ability to determine the initial position of the cube. If you happen to have one of those cameras, put this toggle switch in the On position. Failure to do so will produce the error "Unsolvable cube". You can determine if your camera takes upside-down picures visually, either during the photographing phase, or by pressing the Camera button.
  • Debugging - If enabled, this toggle switch makes the app create a debug PDF file on the device, which is essentially an illustrated log file in which every step of the app's code is documented. Debug PDF files are covered in detail below.
  • Camera Name - Use this drop-down box to select the robot's camera if your device has multiple cameras attached to it. Laptops almost always do.
  • Video Duration - If set to a non-zero value, enables the delayed image capturing mode in which the camera shoots short videos and uses the last frame for analysis instead of taking static photos. Delayed image capturing gives the camera enough time to self-adjust to the surrounding lighting conditions and produce more vivid, easier to analyze images. By default, this value is 1000 ms (1 second). Setting this value to 0 (not recommended) makes the camera take static pictures instead of shooting short videos.

The numerous hue, saturation, brightness and glare-related parameters that can be seen on the right side of the Configuration screen are holdovers from previous versions, and are permanently disabled.

The Area Cutoff, Squareness, Angle Deviation and Size Deviation parameters control the square zone recognition code. The default values have been chosen by trial and error and are generally considered optimal. Changing them is not recommended.


8. Debug PDF Files

The app is designed to provide an insight into its thinking process. When the Debugging toggle switch is on, the app creates a PDF file for every run containing the photographs it has taken, color recognition results, and other information. On Raspberry PI, the PDF files can be found in the folder \Data\Users\DefaultAccount\Pictures. On a regular PC, this folder is This PC\Pictures.

A typical page from the PDF file looks as follows:

The top of the page displays the two photographs taken per cube's face with the zone detection results superimposed over the images. The center square is marked with a dashed outline, the other squares with a solid outline.

The color diagram on the middle left displays the color zones with their hue, saturation and brightness values. This information is generated by the old legacy code and discarded in the Auto-Config mode.

The color diagram on the middle right is the output of the color recognition module in the Auto-Config mode, before error correction is applied. Right below it, error correction log entries may be displayed (not shown here). At the bottom right, the final result, with error correction applied, is displayed.

The debug PDF files are a valuable troubleshooting tool. For example, during a run, the app has generated the error ERROR_UNSOLVABLE_CUBE. The PDF file generated during that run offers an instant explanation for the failure: a mechanical problem caused the cube to be way off center during photographing (see image below). As a result, the color zones were not correctly identified as half of the cube's face was not even within the camera's view.


9. Troubleshooting Tips

RUN Button is Clicked But Nothing Moves

Make sure your Maestro controller' serial mode is set to USB Dual Port via the Pololu Maestro Control Center (Serial Settings tab), and the Apply Settings button is clicked:

Camera not Centered

If the camera is not properly mounted or not pointing at the middle of the cube, a photo may look like the image above . If the app can't see the entire face of the cube, it can't process it. Make sure the center of the cube is roughly in the middle of the photograph. Use the Camera button to take test photos before a run. Adjust the camera's position in the camera holder if necessary. Also make sure the grippers hug the cube tightly to avoid shifting while the cube is being photographed. Adjust servo calibration values if necessary.

Zone Detection Error

If you are running an older version of the app with a high-resolution camera, the robot may display the error ERROR_ZONE_DETECTION_ERROR right after the first face of the cube is photographed, while the debug PDF shows blurry and distorted images:

To fix this problem, please upgrade to the latest version of the app.

Images Appear Upside-Down

Upside-down images are guaranteed to cause the error ERROR_UNSOLVABLE_CUBE accompanied by a plethora of error correction entries in the debug PDF file:

If your camera has this "feature", turn on the Rotate Photos 180° toggle switch on the Configuration screen.

Not Enough Light

If the photographs are too dark, some squares may appear completely black, like the square marked with a red arrow on the image below. This will cause the app to generate the error ERROR_COLOR_COULD_NOT_BE_DETERMINED. Make sure the cube is reasonably well lit, while avoiding glare.

The Colors are Identified Correctly but the Cube is Not Being Resolved

It is critical that your robot photograph the cube's faces in a particular order. Please refer to our introduction video for the correct sequence or rotations during photographing. For example, if you insert the cube with the white center square pointing towards the camera and red center square pointing upwards, then the correct order in which the center squares should appear on the photographs is:

White -> Red -> Yellow ->Orange -> Blue -> Green

If some or all of the gripper servos are calibrated incorrectly and turn from the neutral to 90° position in the wrong direction, the app won't be able to correctly reconstruct the cube's initial position and therefore, won't be able to resolve it.

Please do not hesitate to contact us if your robot keeps refusing to solve the cube. We will do our best to help you.


10. Extra Features: Voice Commands & Random Scrambling

As of Version 3.0.107.0, the app is equipped with a new button, Extras, which takes the user to the Extra Configuration Parameters screen. On this screen, two additional features can be enabled and disabled: Voice Commands and Scrambling. This section describes both features in detail.

10.1 Voice Commands

As of Version 3.0.107.0 of the app, the robot can be started and stopped via voice commands. To enable voice commands, press the EXTRAS button on the main screen, and put the Enable Voice Commands toggle switch in the On position on the Extra Configuration Parameters screen. Enter the phrases for the RUN and STOP commands. In our example, these phrases are "Solve this cube" and "Stop now", but you can pick your own commands.

The Confidence Threshold drop-down box specifies the level of confidence the voice recognition algorithm needs to have to execute a command vs. ignoring it. We recommend setting this option to "Low".

When the Enable Voice Commands option is enabled, an ear symbol is displayed in the status area of the main screen. When a command is received but not recognized, the ear symbol momentarily increases in size and goes back to its normal size.

When the app is running on a Windows 10 PC, the microphone needs to be enabled for the app. To enable it, go to Settings -> Privacy -> Microphone and make sure the toggle switch is on for the RubiksCubeRobot app, as follows:

If the microphone is not enabled, the ear symbol will be replaced by a yellow exclamation sign.

10.2 Random Scrambling

Following your numerous requests, we have equipped the robot with a random scrambling feature. To enable it, press the EXTRAS button on the main screen and put the Enable Scrambling toggle switch in the On position on the Extra Configuration Parameters screen. You also need to specify the number of commands, which should be greater than 0, and optionally a random generator seed value, which is an arbitrary text string.

When scrambling is enabled, the RUN button on the main screen becomes the SCRAMBLE button:

When the SCRAMBLE button is pressed, the robot generates a random sequence of turns, and then executes this sequence. The length of the sequence is specified via the Number of Commands parameter. A single command is a 90°, 180°, or 270° turn of the lower, upper, left, right, front or back face of the cube.

When the Seed Value parameter is empty, the sequence is completely random and unique for every invocation. When the Seed Value is non-empty, it is used as the seed value for the pseudo-random number generator employed by the app. Therefore, a certain seed value generates the same scrambling sequence for every invocation.


<< Hardware