The latest Donkeycar 2 code came be found at: https://github.com/wroscoe/donkey
This code allows you to create a car that can drive an autonomous car using machine learning with Tensorflow.
This code is currently transitioning from Donkeycar to Donkeycar 2. There are some issues if you try to use the latest code from this repository.
So I created a fork of the project with all the fixes needed to run the latest code. You can find the latest code on my forked respository: https://github.com/ricorx7/donkey
Here are the instructions to Install, Setup and Run the Donkeycar 2 code.
You can start with a fresh install of Linux on Desktop and RPI3. Or you can resuse the old RPI3 image and just install over it the new Donkeycar code.
Build
Here is the parts list:
Part Description | Link | Approximate Cost |
---|---|---|
Magnet Car or alternative | $92 | |
M2x6 screws (4) | mcmaster.com/#91292a831/=177k4rp | $6.38 * |
M2.5x12 screws (8) | mcmaster.com/#91292a016/=177k574 | $4.80 * |
M2.5 nuts (8) | mcmaster.com/#91828a113/=177k7ex | $5.64 * |
M2.5 washers (8) | mcmaster.com/#93475a196/=177k7x6 | $1.58 * |
USB Battery with microUSB cable (any battery capable of 2A 5V output is sufficient) | $17 | |
Raspberry Pi 3 | $38 | |
MicroSD Card (many will work, I like this one because it boots quickly) | $18.99 | |
Wide Angle Raspberry Pi Camera | $25 | |
Female to Female Jumper Wire | $7 * | |
Servo Driver PCA 9685 | $12 ** | |
3D Printed roll cage and top plate. | Purchase: Donkey Store Files: thingiverse.com/thing:2260575 |
The full document for the build instructions is here: http://docs.donkeycar.com/
Install
These instructions are for OSX and Linux.
If you have Windows 10, install BASH, this will give you a Linux terminal in Windows 10 and make things a lot easier. Instructions on how to install BASH are here: https://www.howtogeek.com/249966/how-to-install-and-use-the-linux-bash-shell-on-windows-10/
If you have Windows 7, you will need to install python3 and pip.
YOU WILL NEED TO DO THESE STEPS ON BOTH THE RPI3 AND YOUR DESKTOP COMPUTER.
You will need to copy the RPI3 image to a SD card. The instruction to load the SD card can be found here:
https://docs.google.com/document/d/11nu6_ReReoIxA1KVq-sCy7Tczbk6io0izcItucrw7hI/edit#
The code on the image and the python projects are out of date and will need to be updated or reinstalled.
Python
Install virtualenv if you have not done so already.
Lets create a virtual environment
pip install virtualenvSometimes you have to call pip3.
pip3 install virtualenv
Lets create a virtual environment
virtualenv envThis will create a folder env. Now lets activate this virtual environment.
source env/bin/activateYou should see now at the being of your terminal prompt (env) which means you are now working in the virtual environment.
Source Code
Clone the latest version of Donkeycar code. If you want to bypass all the manual sets after this, you can clone my fork which already made these changes.
git clone https://github.com/wroscoe/donkeyOr my code and skip fixing the code:
git clone https://github.com/ricorx7/donkey
This will create a folder donkey wherever you run the command. Go into this folder.
cd donkeyLets let the source code now install all the dependencies with pip install. The "-e" tells pip to install the given project folder.
pip install -e .This will will download all the dependencies and install Donkeycar 2. There were some additional projects that need to be installed.
pip install h5py docoptIf you try to run this code, you will get a error message about "Optimizer with shape(15,)". This is because the "keras" versions do not match between the RPI and the desktop computer. So lets make sure that both the desktop and RPI are running the same version of keras by installing the specific version 2.0.5
pip uninstall kerasOn the desktop computer, you can simply install the latest version of Tensorflow, currently for me it is 1.2.1. RUN THESE COMMANDS ONLY ON THE DESKTOP COMPUTER.
pip install keras==2.0.5
pip install tensorflow --upgrade
On the RPI3, you will need to download the RPI3 version of the latest Tensorflow, version 1.1.0. RUN THESE COMMAND ONLY ON THE RPI3.
wget https://github.com/samjabrahams/tensorflow-on-raspberry-pi/releases/download/v1.1.0/tensorflow-1.1.0-cp34-cp34m-linux_armv7l.whl pip uninstall tensorflow pip install tensorflow-1.1.0-cp34-cp34m-linux_armv7l.whl
Now, lets rebuild Donkeycar 2 with the latest libraries.
pip uninstall donkeycarANYTIME YOU MAKE SOURCE CODE CHANGES, YOU WILL RUN THESE COMMANDS TO REBUILD DONKEYCAR AND INSTALL IT.
pip install -e .
Modify Source Code
There was some issues when i tried to run the original source code. My fork has them fixed and I am trying to get a pull request done to have the wroscoe's code modified. Here are the instructions to fix the code. Make sure you make these changes on both the desktop and RPI3.
keras.py
This file gives an error about 'val_loss' is not a option to use. You will need to change 'val_loss' to 'loss.
Open the file: donkey/donkeycar/parts/ml/keras.py
Search for the:
monitor='val_loss'There are 2 places where it is located. At line 40 and 48. Changes 'val_loss' to 'loss'
monitor='loss'
tub.py
This file gives and error about writing an unknown JSON value 0.0. I added a try/except, to skip it if its a value it does not like. I also output a message, because I am not sure if this needs to be fixed. Currently the throttle does not work when you let it run automous, so it may be here that the issues is located.
Open the file: donkey/donkeycar/parts/stores/tub.py
Go to line 101 'write_json_record()' and add a try/except for 'json.dump()'.
def write_json_record(self, json_data): path = self.get_json_record_path(self.current_ix) with open(path, 'w') as fp: try: json.dump(json_data, fp) except TypeError as te: print('Type Error in tub::write_json_record: ' + str(json_data), te) except Exception as e: print('Exception in tub::write_json_record', e)
Ok, so now the source code can run. So lets rebuild the application.
pip uninstall dockeycar
pip install -e .
Run Code
So now lets get things running. There are 3 steps you will be doing. Driving the car to record images and json files which contains the image, throttle and steering angle. You will then use these recorded data to train a model. You will then drive with this model.
Record Data
Connect the RPI3 to the WIFI connection. Get the IP address of the RPI3.
SSH onto the RPI3 and run this command in the folder that contains the manage.py. This will depend where Donkeycar was installed. The folder that contains manage.py will also contain folders: data, logs, manage.py and models.
ssh pi@192.168.X.XXXThe default password is 'raspberry.
cd donkey
python manage.py drive
This python command is given from the virtual environment. This virtual environment is already created from the image you used from Donkeycar 1.
You will need to know the IP address of the car to view the web interface.
http://192.168.X.XXX:8887/drive
This will display the live video feed. Select a Max Throttle. I choose around 20%.
PS3 Controller
If you have a PS3 game controller, use it, it will make driving a lot easier. If you do not, you can use the blue box as a joystick. If you do have a PS3 game controller, select the "Gamepad" toggle box. You will need to connect the PS3 controller to the desktop or laptop through Bluetooth. Connect the USB cable to the PS3 controller and the desktop. Then turn on the Bluetooth on the desktop computer. Then press the Playstation button on the PS3 controller and unplug the USB connection and see if the Bluetooth connection is made. Press the left joystick on the PS3 controller and see if it moves forward. If it does, then begin driving.Drive the car 20 to 30 laps around the track. Everytime you give the throttle, it will automatically record.
Train Model
Lets get the data off the RPI3 and load it onto the desktop computer. We can then run Tensorflow to train based off the data recorded.
We will use RSYNC to copy the files back and forth between the desktop computer and RPI3. It will only copy over changes after the initial copy.
Lets make a directory to store the data off the RPI3. I created a folder in the same folder as env for the virtual environment.
To copy data off the RPI3 to the desktop. The donkey folder, should be the folder with: data, logs, manage.py, models. If 'donkey' was not the folder, then set the correct folder is the first path.
This will copy all the data from the RPI3 to the folder rpi.
You may have to the give the full path to the tub_XXX folder.
This will begin Tensorflow and start training based off the data. Eventually it will stop running. This is based off the settings in the keras.py folder in the Donkeycar project. Look for the function "early_stop = keras.callbacks.EarlyStopping" around line 46.
When it completes, you will have a new file in the folder 'models' called myModel.
Lets copy this file back to the RPI
This should only copy over 1 file, the new myModel file.
Now its time to test this myModel
Open the web browser on the desktop. Connect your PS3 controller. Set you max throttle to the same throttle you set when recording the data.
Select "Local Angle (d)" under "Mode & Pilot".
Throttle up on the PS3 controller and verify the car will now steer on its own. You are now one step closer to an autonomous vehicle.
Now i need to figure out how to make the throttle also works. Also, there is code to control the car using the PS3 controller directly on the car instead of going through the WIFI. This will fix the latency between the PS3 controller and the car.
https://gist.github.com/tawnkramer/8477071f176bf476f06db55a1d650580
We will use RSYNC to copy the files back and forth between the desktop computer and RPI3. It will only copy over changes after the initial copy.
Lets make a directory to store the data off the RPI3. I created a folder in the same folder as env for the virtual environment.
mkdir rpi
To copy data off the RPI3 to the desktop. The donkey folder, should be the folder with: data, logs, manage.py, models. If 'donkey' was not the folder, then set the correct folder is the first path.
rsync -ah --progress pi@192.168.X.XX/donkey rpi
This will copy all the data from the RPI3 to the folder rpi.
cd rpiGo into the folder and now begin the training.
python manage.py train --model=myModel --tub=rpi/data/tub_3_XXXX
You may have to the give the full path to the tub_XXX folder.
This will begin Tensorflow and start training based off the data. Eventually it will stop running. This is based off the settings in the keras.py folder in the Donkeycar project. Look for the function "early_stop = keras.callbacks.EarlyStopping" around line 46.
When it completes, you will have a new file in the folder 'models' called myModel.
Lets copy this file back to the RPI
rsync -ah --progress rpi pi@192.168.X.XXX/donkey
This should only copy over 1 file, the new myModel file.
Now its time to test this myModel
Run the Model
Go back to the RPI3 and lets drive again. This time it will steer on its own, but you will have to control the throttle. Currently the code does not work for throttle.python manage.py drive -model=models/myModel
Open the web browser on the desktop. Connect your PS3 controller. Set you max throttle to the same throttle you set when recording the data.
Select "Local Angle (d)" under "Mode & Pilot".
Throttle up on the PS3 controller and verify the car will now steer on its own. You are now one step closer to an autonomous vehicle.
Now i need to figure out how to make the throttle also works. Also, there is code to control the car using the PS3 controller directly on the car instead of going through the WIFI. This will fix the latency between the PS3 controller and the car.
https://gist.github.com/tawnkramer/8477071f176bf476f06db55a1d650580
Thanks for the guide, it clarified a lot of my questions. I'm just got done building one and was reading on the Slack channel about the training and found this post. Thanks for sharing your experience. I found a way to use a webcam instead of the RPi cam.
ReplyDeleteHi Damaso Cardenales. I have tried to change to an usb camera, but I have not succeeded. Could you pass me your code? that would help me a lot.
DeleteThanks for the guide, I need to know if I use normale dc motors not the 390 ones, how can I still follow the guide? since the dc doesn't have the steering things just different voltages.
ReplyDeleteThe "Servo Driver PCA 9685" relays the steering angle to the RPI3. The steering is controlled by a servo signal from the remote control. The remote signal is then passed into the servo driver. The servo driver then allows the RPI3 to monitor to the angle and also pass it to the motor controller to also steer the car.
DeleteHow do I change the number of epochs to train the model?
ReplyDeleteLook in the file Parts->ml->keras.py. There is the epoch parameter.
DeleteYoure so cool! I dont suppose Ive read anything like this before. So nice to find somebody with some original thoughts on this subject. realy thank you for starting this up. this website is something that is needed on the web, someone with a little originality. useful job for bringing something new to the internet!
ReplyDeleteBEST BABY MONITOR REVIEWS 2018
Nice to read your article! I am looking forward to sharing your adventures and experiences. dvd duplication services
ReplyDelete4WD contract vehicles can go on most streets in Australia however there will even now be some boycotted tracks that are denied. car dealerships near me
ReplyDeleteThis comment has been removed by the author.
ReplyDeleteThis comment has been removed by the author.
ReplyDeleteThe majority of tractor accidents come from what are known as overturns and run overs. Overtones are where the tractor for a number of reasons will tilt on its side to such an angle that it falls over. compact tractor attachments
ReplyDelete