Monday, August 21, 2017

Donkeycar 2 - Install, Setup and Run an Autonomous Vehicle




The latest Donkeycar 2 code came be found at: https://github.com/wroscoe/donkey

This code allows you to create a car that can drive an autonomous car using machine learning with Tensorflow.

This code is currently transitioning from Donkeycar to Donkeycar 2.  There are some issues if you try to use the latest code from this repository.

So I created a fork of the project with all the fixes needed to run the latest code.  You can find the latest code on my forked respository: https://github.com/ricorx7/donkey

Here are the instructions to Install, Setup and Run the Donkeycar 2 code.

You can start with a fresh install of Linux on Desktop and RPI3.  Or you can resuse the old RPI3 image and just install over it the new Donkeycar code.

Build

Here is the parts list:
Part DescriptionLinkApproximate Cost
Magnet Car or alternative$92
M2x6 screws (4)mcmaster.com/#91292a831/=177k4rp$6.38 *
M2.5x12 screws (8)mcmaster.com/#91292a016/=177k574$4.80 *
M2.5 nuts (8)mcmaster.com/#91828a113/=177k7ex$5.64 *
M2.5 washers (8)mcmaster.com/#93475a196/=177k7x6$1.58 *
USB Battery with microUSB cable (any battery capable of 2A 5V output is sufficient)$17
Raspberry Pi 3$38
MicroSD Card (many will work, I like this one because it boots quickly)$18.99
Wide Angle Raspberry Pi Camera$25
Female to Female Jumper Wire$7 *
Servo Driver PCA 9685$12 **
3D Printed roll cage and top plate.Purchase: Donkey Store Files: thingiverse.com/thing:2260575



The full document for the build instructions is here: http://docs.donkeycar.com/


Install

These instructions are for OSX and Linux.  

If you have Windows 10, install BASH, this will give you a Linux terminal in Windows 10 and make things a lot easier.   Instructions on how to install BASH are here: https://www.howtogeek.com/249966/how-to-install-and-use-the-linux-bash-shell-on-windows-10/

If you have Windows 7, you will need to install python3 and pip. 


YOU WILL NEED TO DO THESE STEPS ON BOTH THE RPI3 AND YOUR DESKTOP COMPUTER.

You will need to copy the RPI3 image to a SD card.  The instruction to load the SD card can be found here:
https://docs.google.com/document/d/11nu6_ReReoIxA1KVq-sCy7Tczbk6io0izcItucrw7hI/edit#
The code on the image and the python projects are out of date and will need to be updated or reinstalled.

Python

Install virtualenv if you have not done so already.
pip install virtualenv
Sometimes you have to call pip3.
pip3 install virtualenv 

Lets create a virtual environment

virtualenv env
This will create a folder env. Now lets activate this virtual environment.
source env/bin/activate
You should see now at the being of your terminal prompt (env) which means you are now working in the virtual environment.


Source Code

Clone the latest version of Donkeycar code.  If you want to bypass all the manual sets after this, you can clone my fork which already made these changes.

git clone https://github.com/wroscoe/donkey
Or my code and skip fixing the code:
git clone https://github.com/ricorx7/donkey


This will create a folder donkey wherever you run the command.  Go into this folder.
cd donkey 
Lets let the source code now install all the dependencies with pip install.  The "-e" tells pip to install the given project folder.
pip install -e .
This will will download all the dependencies and install Donkeycar 2.  There were some additional projects that need to be installed.
pip install h5py docopt
If you try to run this code, you will get a error message about "Optimizer with shape(15,)".  This is because the "keras" versions do not match between the RPI and the desktop computer.  So lets make sure that both the desktop and RPI are running the same version of keras by installing the specific version 2.0.5
pip uninstall keras
pip install keras==2.0.5
On the desktop computer, you can simply install the latest version of Tensorflow, currently for me it is 1.2.1.  RUN THESE COMMANDS ONLY ON THE DESKTOP COMPUTER.
pip install tensorflow --upgrade


On the RPI3, you will need to download the RPI3 version of the latest Tensorflow, version 1.1.0.  RUN THESE COMMAND ONLY ON THE RPI3.

wget https://github.com/samjabrahams/tensorflow-on-raspberry-pi/releases/download/v1.1.0/tensorflow-1.1.0-cp34-cp34m-linux_armv7l.whl
pip uninstall tensorflow
pip install tensorflow-1.1.0-cp34-cp34m-linux_armv7l.whl

Now, lets rebuild Donkeycar 2 with the latest libraries.
pip uninstall donkeycar
pip install -e .
ANYTIME YOU MAKE SOURCE CODE CHANGES, YOU WILL RUN THESE COMMANDS TO REBUILD DONKEYCAR AND INSTALL IT.


Modify Source Code

There was some issues when i tried to run the original source code.  My fork has them fixed and I am trying to get a pull request done to have the wroscoe's code modified.  Here are the instructions to fix the code.  Make sure you make these changes on both the desktop and RPI3.

keras.py

This file gives an error about 'val_loss' is not a option to use.  You will need to change 'val_loss' to 'loss.

Open the file: donkey/donkeycar/parts/ml/keras.py
Search for the:
monitor='val_loss'
There are 2 places where it is located.  At line  40 and 48.  Changes 'val_loss' to 'loss'
monitor='loss'

tub.py

This file gives and error about writing an unknown JSON value 0.0.  I added a try/except, to skip it if its a value it does not like.  I also output a message, because I am not sure if this needs to be fixed.  Currently the throttle does not work when you let it run automous, so it may be here that the issues is located.

Open the file: donkey/donkeycar/parts/stores/tub.py
Go to line 101 'write_json_record()' and add a try/except for 'json.dump()'.
    def write_json_record(self, json_data):
        path = self.get_json_record_path(self.current_ix)
        with open(path, 'w') as fp:
            try:
                json.dump(json_data, fp)
            except TypeError as te:
                print('Type Error in tub::write_json_record: ' + str(json_data), te)
            except Exception as e:
                print('Exception in tub::write_json_record', e)



Ok, so now the source code can run.  So lets rebuild the application.
pip uninstall dockeycar
pip install -e .

Run Code

So now lets get things running.  There are 3 steps you will be doing.  Driving the car to record images and json files which contains the image, throttle and steering angle.  You will then use these recorded data to train a model.  You will then drive with this model.

Record Data

Connect the RPI3 to the WIFI connection.  Get the IP address of the RPI3.

SSH onto the RPI3 and run this command in the folder that contains the manage.py.  This will depend where Donkeycar was installed.  The folder that contains manage.py will also contain folders: data, logs, manage.py and models.

ssh pi@192.168.X.XXX
The default password is 'raspberry.


cd donkey
python manage.py drive

This python command is given from the virtual environment.  This virtual environment is already created from the image you used from Donkeycar 1.

You will need to know the IP address of the car to view the web interface.
http://192.168.X.XXX:8887/drive

This will display the live video feed.  Select a Max Throttle.  I choose around 20%.

PS3 Controller

If you have a PS3 game controller, use it, it will make driving a lot easier.  If you do not, you can use the blue box as a joystick.  If you do have a PS3 game controller, select the "Gamepad" toggle box.  You will need to connect the PS3 controller to the desktop or laptop through Bluetooth.  Connect the USB cable to the PS3 controller and the desktop.  Then turn on the Bluetooth on the desktop computer.  Then press the Playstation button on the PS3 controller and unplug the USB connection and see if the Bluetooth connection is made.  Press the left joystick on the PS3 controller and see if it moves forward.  If it does, then begin driving.

Drive the car 20 to 30 laps around the track.  Everytime you give the throttle, it will automatically record.


Train Model

Lets get the data off the RPI3 and load it onto the desktop computer.  We can then run Tensorflow to train based off the data recorded.

We will use RSYNC to copy the files back and forth between the desktop computer and RPI3.  It will only copy over changes after the initial copy.

Lets make a directory to store the data off the RPI3.  I created a folder in the same folder as env for the virtual environment.

mkdir rpi

To copy data off the RPI3 to the desktop.  The donkey folder, should be the folder with: data, logs, manage.py, models.  If 'donkey' was not the folder, then set the correct folder is the first path.

rsync -ah --progress pi@192.168.X.XX/donkey  rpi

This will  copy all the data from the RPI3 to the folder rpi.

cd rpi
Go into the folder and now begin the training.

python manage.py train --model=myModel --tub=rpi/data/tub_3_XXXX

You may have to the give the full path to the tub_XXX folder.

This will begin Tensorflow and start training based off the data.  Eventually it will stop running.  This is based off the settings in the keras.py folder in the Donkeycar project.  Look for the function "early_stop = keras.callbacks.EarlyStopping" around line 46.

When it completes, you will have a new file in the folder 'models' called myModel.

Lets copy this file back to the RPI
rsync -ah --progress rpi pi@192.168.X.XXX/donkey

This should only copy over 1 file, the new myModel file.
Now its time to test this myModel

Run the Model

Go back to the RPI3 and lets drive again.  This time it will steer on its own, but you will have to control the throttle.  Currently the code does not work for throttle.

python manage.py drive -model=models/myModel

Open the web browser on the desktop.  Connect your PS3 controller.  Set you max throttle to the same throttle you set when recording the data.

Select "Local Angle (d)" under "Mode & Pilot".

Throttle up on the PS3 controller and verify the car will now steer on its own.  You are now one step closer to an autonomous vehicle.


Now i need to figure out how to make the throttle also works.  Also, there is code to control the car using the PS3 controller directly on the car instead of going through the WIFI.  This will fix the latency between the PS3 controller and the car.
https://gist.github.com/tawnkramer/8477071f176bf476f06db55a1d650580


11 comments:

  1. Thanks for the guide, it clarified a lot of my questions. I'm just got done building one and was reading on the Slack channel about the training and found this post. Thanks for sharing your experience. I found a way to use a webcam instead of the RPi cam.

    ReplyDelete
    Replies
    1. Hi Damaso Cardenales. I have tried to change to an usb camera, but I have not succeeded. Could you pass me your code? that would help me a lot.

      Delete
  2. Thanks for the guide, I need to know if I use normale dc motors not the 390 ones, how can I still follow the guide? since the dc doesn't have the steering things just different voltages.

    ReplyDelete
    Replies
    1. The "Servo Driver PCA 9685" relays the steering angle to the RPI3. The steering is controlled by a servo signal from the remote control. The remote signal is then passed into the servo driver. The servo driver then allows the RPI3 to monitor to the angle and also pass it to the motor controller to also steer the car.

      Delete
  3. How do I change the number of epochs to train the model?

    ReplyDelete
    Replies
    1. Look in the file Parts->ml->keras.py. There is the epoch parameter.

      Delete
  4. Youre so cool! I dont suppose Ive read anything like this before. So nice to find somebody with some original thoughts on this subject. realy thank you for starting this up. this website is something that is needed on the web, someone with a little originality. useful job for bringing something new to the internet!
    BEST BABY MONITOR REVIEWS 2018

    ReplyDelete
  5. Nice to read your article! I am looking forward to sharing your adventures and experiences. dvd duplication services

    ReplyDelete
  6. 4WD contract vehicles can go on most streets in Australia however there will even now be some boycotted tracks that are denied. car dealerships near me

    ReplyDelete
  7. This comment has been removed by the author.

    ReplyDelete
  8. This comment has been removed by the author.

    ReplyDelete