The latest Donkeycar 2 code came be found at:
https://github.com/wroscoe/donkey
This code allows you to create a car that can drive an autonomous car using machine learning with Tensorflow.
This code is currently transitioning from Donkeycar to Donkeycar 2. There are some issues if you try to use the latest code from this repository.
So I created a fork of the project with all the fixes needed to run the latest code. You can find the latest code on my forked respository:
https://github.com/ricorx7/donkey
Here are the instructions to Install, Setup and Run the Donkeycar 2 code.
You can start with a fresh install of Linux on Desktop and RPI3. Or you can resuse the old RPI3 image and just install over it the new Donkeycar code.
Build
Here is the parts list:
The full document for the build instructions is here:
http://docs.donkeycar.com/
Install
These instructions are for OSX and Linux.
If you have Windows 7, you will need to install python3 and pip.
YOU WILL NEED TO DO THESE STEPS ON BOTH THE RPI3 AND YOUR DESKTOP COMPUTER.
Python
Install virtualenv if you have not done so already.
pip install virtualenv
Sometimes you have to call pip3.
pip3 install virtualenv
Lets create a virtual environment
virtualenv env
This will create a folder env. Now lets activate this virtual environment.
source env/bin/activate
You should see now at the being of your terminal prompt (env) which means you are now working in the virtual environment.
Source Code
Clone the latest version of Donkeycar code. If you want to bypass all the manual sets after this, you can clone my fork which already made these changes.
git clone https://github.com/wroscoe/donkey
Or my code and skip fixing the code:
git clone https://github.com/ricorx7/donkey
This will create a folder donkey wherever you run the command. Go into this folder.
cd donkey
Lets let the source code now install all the dependencies with pip install. The "-e" tells pip to install the given project folder.
pip install -e .
This will will download all the dependencies and install Donkeycar 2. There were some additional projects that need to be installed.
pip install h5py docopt
If you try to run this code, you will get a error message about "Optimizer with shape(15,)". This is because the "keras" versions do not match between the RPI and the desktop computer. So lets make sure that both the desktop and RPI are running the same version of keras by installing the specific version 2.0.5
pip uninstall keras
pip install keras==2.0.5
On the desktop computer, you can simply install the latest version of Tensorflow, currently for me it is 1.2.1.
RUN THESE COMMANDS ONLY ON THE DESKTOP COMPUTER.
pip install tensorflow --upgrade
On the RPI3, you will need to download the RPI3 version of the latest Tensorflow, version 1.1.0.
RUN THESE COMMAND ONLY ON THE RPI3.
wget https://github.com/samjabrahams/tensorflow-on-raspberry-pi/releases/download/v1.1.0/tensorflow-1.1.0-cp34-cp34m-linux_armv7l.whl
pip uninstall tensorflow
pip install tensorflow-1.1.0-cp34-cp34m-linux_armv7l.whl
Now, lets rebuild Donkeycar 2 with the latest libraries.
pip uninstall donkeycar
pip install -e .
ANYTIME YOU MAKE SOURCE CODE CHANGES, YOU WILL RUN THESE COMMANDS TO REBUILD DONKEYCAR AND INSTALL IT.
Modify Source Code
There was some issues when i tried to run the original source code. My fork has them fixed and I am trying to get a pull request done to have the wroscoe's code modified. Here are the instructions to fix the code. Make sure you make these changes on both the desktop and RPI3.
keras.py
This file gives an error about 'val_loss' is not a option to use. You will need to change 'val_loss' to 'loss.
Open the file: donkey/donkeycar/parts/ml/keras.py
Search for the:
monitor='val_loss'
There are 2 places where it is located. At line 40 and 48. Changes 'val_loss' to 'loss'
monitor='loss'
tub.py
This file gives and error about writing an unknown JSON value 0.0. I added a try/except, to skip it if its a value it does not like. I also output a message, because I am not sure if this needs to be fixed. Currently the throttle does not work when you let it run automous, so it may be here that the issues is located.
Open the file: donkey/donkeycar/parts/stores/tub.py
Go to line 101 'write_json_record()' and add a try/except for 'json.dump()'.
def write_json_record(self, json_data):
path = self.get_json_record_path(self.current_ix)
with open(path, 'w') as fp:
try:
json.dump(json_data, fp)
except TypeError as te:
print('Type Error in tub::write_json_record: ' + str(json_data), te)
except Exception as e:
print('Exception in tub::write_json_record', e)
Ok, so now the source code can run. So lets rebuild the application.
pip uninstall dockeycar
pip install -e .
Run Code
So now lets get things running. There are 3 steps you will be doing. Driving the car to record images and json files which contains the image, throttle and steering angle. You will then use these recorded data to train a model. You will then drive with this model.
Record Data
Connect the RPI3 to the WIFI connection. Get the IP address of the RPI3.
SSH onto the RPI3 and run this command in the folder that contains the manage.py. This will depend where Donkeycar was installed. The folder that contains manage.py will also contain folders: data, logs, manage.py and models.
ssh pi@192.168.X.XXX
The default password is 'raspberry.
cd donkey
python manage.py drive
This python command is given from the virtual environment. This virtual environment is already created from the image you used from Donkeycar 1.
You will need to know the IP address of the car to view the web interface.
http://192.168.X.XXX:8887/drive
This will display the live video feed. Select a Max Throttle. I choose around 20%.
PS3 Controller
If you have a PS3 game controller, use it, it will make driving a lot easier. If you do not, you can use the blue box as a joystick. If you do have a PS3 game controller, select the "Gamepad" toggle box. You will need to connect the PS3 controller to the desktop or laptop through Bluetooth. Connect the USB cable to the PS3 controller and the desktop. Then turn on the Bluetooth on the desktop computer. Then press the Playstation button on the PS3 controller and unplug the USB connection and see if the Bluetooth connection is made. Press the left joystick on the PS3 controller and see if it moves forward. If it does, then begin driving.
Drive the car 20 to 30 laps around the track. Everytime you give the throttle, it will automatically record.
Train Model
Lets get the data off the RPI3 and load it onto the desktop computer. We can then run Tensorflow to train based off the data recorded.
We will use RSYNC to copy the files back and forth between the desktop computer and RPI3. It will only copy over changes after the initial copy.
Lets make a directory to store the data off the RPI3. I created a folder in the same folder as env for the virtual environment.
mkdir rpi
To copy data off the RPI3 to the desktop. The donkey folder, should be the folder with: data, logs, manage.py, models. If 'donkey' was not the folder, then set the correct folder is the first path.
rsync -ah --progress pi@192.168.X.XX/donkey rpi
This will copy all the data from the RPI3 to the folder rpi.
cd rpi
Go into the folder and now begin the training.
python manage.py train --model=myModel --tub=rpi/data/tub_3_XXXX
You may have to the give the full path to the tub_XXX folder.
This will begin Tensorflow and start training based off the data. Eventually it will stop running. This is based off the settings in the keras.py folder in the Donkeycar project. Look for the function "early_stop = keras.callbacks.EarlyStopping" around line 46.
When it completes, you will have a new file in the folder 'models' called myModel.
Lets copy this file back to the RPI
rsync -ah --progress rpi pi@192.168.X.XXX/donkey
This should only copy over 1 file, the new myModel file.
Now its time to test this myModel
Run the Model
Go back to the RPI3 and lets drive again. This time it will steer on its own, but you will have to control the throttle. Currently the code does not work for throttle.
python manage.py drive -model=models/myModel
Open the web browser on the desktop. Connect your PS3 controller. Set you max throttle to the same throttle you set when recording the data.
Select "Local Angle (d)" under "Mode & Pilot".
Throttle up on the PS3 controller and verify the car will now steer on its own. You are now one step closer to an autonomous vehicle.
Now i need to figure out how to make the throttle also works. Also, there is code to control the car using the PS3 controller directly on the car instead of going through the WIFI. This will fix the latency between the PS3 controller and the car.
https://gist.github.com/tawnkramer/8477071f176bf476f06db55a1d650580
