CARLA (Car Learning to Act) is an open-source platform for developing, training and evaluating autonomous driving agents, built on top of Unreal Engine.

North stars



watod -t gui_tools
rosrun carla_config

CARLA Synthetic Data Generation

Writing notes about how to use CARLA with a focus on how we use this to generate synthetic data.

Asynchronous vs. Synchronous Mode

  • Asynchronous mode (DEFAULT) Server runs the simulation as fast as possible, without waiting for the client
  • Synchronous mode Server waits for a client tick, a “ready to go” message, before updating to the following simulation step.

Variable Time-Step vs. Fixed Time-Step

  • Variable time-step (DEFAULT) The simulation time that goes by between steps will be the time that the server takes to compute these
  • Fixed time-step The elapsed time remains constant between steps


In Synchronous mode, always USE Fixed Time-Step.

If you want things to run as fast as possible, use Asynchronous + Fixed Time-Step

  • Why not variable time-step? This is like: however long it takes to render the next frame, CARLA will take its time. But with fixed time-step, it’s like an impending deadline, i.e. you only have 0.1 seconds to render this, do whatever you can.

CARLA Recorder


Actors vs. Blueprints

Actors: elements that perform actions within the simulation, and they can affect other actors. Actors in CARLA includes

  • vehicles
  • walkers
  • sensors,
  • traffic signs and traffic lights
  • the spectator

So basically, everything we spawn in an actor.

Blueprints are layouts that allow the user to smoothly incorporate new actors into the simulation. They are already-made models with animations and a series of attributes.

You can get a particular blueprint from the blueprint library.

But generally, it seems that we just directly choose an actor from the blueprint library, like the following

vehicle_bp = random.choice(blueprint_library.filter('vehicle.*.*'))

Life Cycle:

  • Spawning
  • Handling
  • Destruction

Coordinate System

CARLA uses the Unreal Engine coordinates system. Remember that carla.Rotation constructor is defined as (pitch, yaw, roll), that differs from Unreal Engine Editor (roll, pitch, yaw).



  • Town01: A basic town layout consisting of “T junctions”.

  • Town02: Similar to Town01, but smaller.

  • Town03: The most complex town, with a 5-lane junction, a roundabout, unevenness, a tunnel, and more.

  • Town04: An infinite loop with a highway and a small town.

  • Town05: Squared-grid town with cross junctions and a bridge. It has multiple lanes per direction. Useful to perform lane changes.

  • Town06: Long highways with many highway entrances and exits. It also has a Michigan left.

  • Town07: A rural environment with narrow roads, barns and hardly any traffic lights.

  • Town10: A city environment with different environments such as an avenue or promenade, and more realistic textures.

CARLA attachments: You can choose between a rigid attachment, and a spring_arm attachment.

CARLA Sensors:

So then when you spawn with attachment, location is specified relative to the parent actor.

transform = carla.Transform(carla.Location(x=0.8, z=1.7))
sensor = world.spawn_actor(blueprint, transform, attach_to=my_vehicle)

How to generate data? Scenario runner is an option:

Scenario Runner

Traffic manager:

The Traffic Manager (TM) is the module that controls vehicles in autopilot mode in a simulation. Its goal is to populate a simulation with realistic urban traffic conditions. Users can customize some behaviors, for example, to set specific learning circumstances.

Other repo, though this uses an older version of CARLA:

  • However, much better learning if you write your things from scratch

Understand how maps work:


List of sensors:

CARLA AD Challenge

“Sensor data is also available through ROS topics. The topic name is structured as follows: /carla/hero/<sensor-id>. For a complete reference check the CARLA ROS sensor documentation"".


Carla ros bridge stuff:

  • It seems that we write some custom things. so that it publishes to /obstacles_3d. this is not out of the box behavior.
  • Why cannot we just use this thing? Get all other actors

This is the key function

def get_data(self, filters):
	Get data specifically from Carla.
		filters: list - list of strings that name Carla-specific filters
			e.g. ["filter_facing_ego"] for traffic signs
	Return Type:
		Dict - eg {CARLA_3D_OBSTACLES: obs_list_ros_msg_obj}
	actors =
	ts_list = TrafficSignListMsg()
	ts_list.header.frame_id = 'odom'
	ts_list.header.stamp =
	traffic_actors = actors.filter("traffic.*")
	if "filter_facing_ego" in filters:
		# only send stop signs that are facing the ego to emulate real life
		applicable_traffic_signs = filter_facing_ego(
	ts_list.traffic_signs = from_carla_traffic_signs(applicable_traffic_signs)
	obs_list = ObstacleListMsg()
	obs_list.header.frame_id = 'odom'
	obs_list.header.stamp =
	obs_list.obstacles = actors_to_obstacles(actors, self.ego_vehicle)
	return {
		CARLA_3D_OBSTACLES: obs_list,

If you want to do manual control, you should go into the gui_tools docker container, and then launch roslaunch carla_manual_control carla_manual_control.launch. You need to press b to enable manual override, and then use the wasd keys to control the car.


watod up carla_server
watod up carla_ros_bridge gui_tools # In a separate terminal

Some extra notes

  • Rowan was telling me about the odom errors we were running into. You should first start the carla server, which runs on unreal engine. Then, you should launch the carla_ros_bridge, which is essentially the frontend.
  • so you can start with
  • watod up carla_server (the backend)
  • Then, do watod up carla_ros_bridge (the frontend)
  • And other instances

The error I was getting gui_tools_1 | [ERROR] [1668642643.334595325, 6.032078996]: Error getting latest time from frame 'base_link' to frame 'odom': Could not find a connection between 'odom' and 'base_link' because they are not part of the same tree.Tf has two or more unconnected trees. (Error code: 2), is because the carla server was not launched yet, the the gui_tools is fetching this transform which is not provided.

The error you were getting from gui_tools was due to a particular transform not being provided.

Installation in Docker

First pull the carla image by running

docker pull carlasim/carla:0.9.13