Traffic Sign Assist, function - GF54.30-P-0010GN

MODEL 166 as of model year 2016 with CODE 513 (Speed Limit Assist) 

MODEL 292 with CODE 513 (Speed Limit Assist) 

Block diagram, except model 166.063 

Block diagram, model 166.063 

Function requirements, general 

• Circuit 15R ON

The electronic ignition lock control unit sends the status of circuit 15 via the chassis CAN 2, chassis gateway control unit (with CODE 23P (Driving assistance package plus)), and chassis FlexRay to the stereo multifunction camera (with CODE 23P (Driving assistance package plus)) or via the chassis CAN 2 to the multifunction camera (with CODE 22P (Lane package)).

Traffic Sign Assist, general 

A camera on the inside of the windshield detects the signposted speed limits, e.g. at the edge of the road, on overhead signs or at construction sites. Their data is compared with the information of the navigation system and displayed both in the instrument cluster and in the navigation map view. "No passing" zones and the associated derestriction signs are also recorded and displayed. For no-entry zones with corresponding signage, a visual and acoustic warning is output in the instrument cluster.

MODEL 166 as of model year 2016 with CODE 494 (US version) with CODE 513 (Speed Limit Assist)

MODEL 292 with CODE 494 (US version) with CODE 513 (Speed Limit Assist)

Here, the stereo multifunction camera (with CODE 23P (Driving assistance package plus)) serves only as a gateway and for display logic purposes (supplementary signs referenced in the map are evaluated), since the traffic signs themselves cannot be detected by it. The values that are defined in the map are displayed in "look and feel" style (USA).

Traffic Sign Assist is implemented in the multifunction camera (with CODE 22P (Lane package)) or the stereo multifunction camera (with CODE 23P (Driving assistance package plus)), depending on equipment.

The following traffic signs are recorded and evaluated on the stretch of road being driven on by the vehicle:

• Speed limits including additional signs along with the corresponding termination signs
• No overtaking signs and their termination
• No entry signs

The detected traffic signs are shown in the multifunction display of the instrument cluster and also in the Audio/COMAND display. The display in the multifunction display of the instrument cluster can be switched on or off using the "Assistance" menu of the instrument cluster.

Display in instrument cluster (depiction of the detected traffic sign is country-specific) 

A maximum of two speed limits and an additional traffic sign are shown at the same time. If several speed limits are detected, the lowest speed limit is displayed.

Display in Audio/COMAND display, system language German (depiction of the detected traffic sign is country-specific) 

The display in the Audio/COMAND display is only shown in the Navigation system map view, and it can be switched on or off using the menu in the COMAND controller unit.

The multifunction camera (with CODE 22P (Lane package)) or stereo multifunction camera (with CODE 23P (Driving assistance package plus)) evaluates the following input factors, among other information: 

• Signal "Engine running" or "Drive train operational"
• Map data
• Direction of travel, vehicle speed and yaw rate
• Steering wheel angle
• Windshield wiper active
• Outside temperature
• Trailer recognition (with CODE 550 (Trailer hitch) (except model 166.063))
• Status of combination switch (set the turn signal indicator at the combination switch)

"Engine running" or "Drive train operational" signal:

The CDI control unit or ME-SFI [ME] control unit sends the "engine running" or "drivetrain operational" signal via the chassis CAN 1, chassis gateway control unit (with CODE 23P (Driving assistance package plus)), and chassis FlexRay to the stereo multifunction camera (with CODE 23P (Driving assistance package plus)) or via the chassis CAN 1, electronic ignition lock control unit, and chassis CAN 2 to the multifunction camera (with CODE 22P (Lane package)).

The ME-SFI [ME] control unit sends the "engine running" or "drivetrain operational" signal via the engine CAN, powertrain control unit, chassis CAN 1, chassis gateway control unit (with CODE 23P (Driving assistance package plus)), and chassis FlexRay to the stereo multifunction camera (with CODE 23P (Driving assistance package plus)) or via the chassis CAN 1, electronic ignition lock control unit, and chassis CAN 2 to the multifunction camera (with CODE 22P (Lane package)).

Map data:

The map data serves to check the object recorded by the multifunction camera for plausibility as well as display speed limits (e.g. 50 km/h) stored in the map when driving into built-up areas. The map data is sent by the COMAND controller unit via the interior CAN, electronic ignition lock control unit and chassis CAN 1, chassis gateway control unit (with CODE 23P (Driving assistance package plus)), and chassis FlexRay to the stereo multifunction camera (with CODE 23P (Driving assistance package plus)) or via the interior CAN, electronic ignition lock control unit, and chassis CAN 2 to the multifunction camera (with CODE 22P (Lane package)).

Direction of travel, vehicle speed and yaw rate:

Information on direction of travel and vehicle speed are derived from the wheel speed and wheel rotation direction signals.

The yaw behavior is registered by the turn rate sensor for lateral and longitudinal acceleration. The yaw rate sensor for lateral and longitudinal acceleration transmits appropriate values via the dynamics CAN to the Electronic Stability Program control unit.

The Electronic Stability Program control unit sends all relevant information via the chassis CAN 1, chassis gateway control unit (with CODE 23P (Driving assistance package plus)), and chassis FlexRay to the stereo multifunction camera (with CODE 23P (Driving assistance package plus)) or via the chassis CAN 1, electronic ignition lock control unit, and chassis CAN 2 to the multifunction camera (with CODE 22P (Lane package)).

Steering wheel angle:

The steering angle is recorded by the steering wheel angle sensor. The steering column tube module control unit directly reads in the signals from the steering wheel angle sensor and sends information about the steering wheel angle via the chassis CAN 1, chassis gateway control unit (with CODE 23P (Driving assistance package plus)), and chassis FlexRay to the stereo multifunction camera (with CODE 23P (Driving assistance package plus)) or via the chassis CAN 1, electronic ignition lock control unit, and chassis CAN 2 to the multifunction camera (with CODE 22P (Lane package)).

Windshield wiper active:

The SAM control unit controls the windshield wiper system. When the wipers are switched on, the SAM control unit sends the "wipers active" status via the interior CAN, electronic ignition lock control unit, chassis CAN 1, chassis gateway control unit (with CODE 23P (Driving assistance package plus)), and chassis FlexRay to the stereo multifunction camera (with CODE 23P (Driving assistance package plus)) or via the interior CAN, electronic ignition lock control unit, and chassis CAN 2 to the multifunction camera (with CODE 22P (Lane package)).

Outside temperature:

The outside temperature is recorded by the outside temperature sensor. The SAM control unit reads in the outside temperature sensor signal directly, uses this to calculate the outside temperature and sends it via interior CAN, electronic ignition lock control unit and chassis CAN 2 to the instrument cluster. The instrument cluster calculates the outside temperature displayed and sends this information via the chassis CAN 2, chassis gateway control unit (with CODE 23P (Driving assistance package plus)), and chassis FlexRay to the stereo multifunction camera (with CODE 23P (Driving assistance package plus)) or via the chassis CAN 2 to the multifunction camera (with CODE 22P (Lane package)).

Trailer recognition (with CODE (550) Trailer hitch) (except model 166.063):

If a trailer is detected, the trailer recognition control unit (with CODE 550 (Trailer hitch)) sends the "trailer detected" status via the interior CAN, electronic ignition lock control unit and chassis CAN 1, chassis gateway control unit (with CODE 23P (Driving assistance package plus)), and chassis FlexRay to the stereo multifunction camera (with CODE 23P (Driving assistance package plus)) or via the interior CAN, electronic ignition lock control unit, and chassis CAN 2 to the multifunction camera (with CODE 22P (Lane package)).

Status of combination switch (setting the turn signal indicator at the combination switch):

The status of the combination switch is directly read in by the steering column tube module control unit and sent via the chassis CAN 1, chassis gateway control unit (with CODE 23P (Driving assistance package plus)), and chassis FlexRay to the stereo multifunction camera (with CODE 23P (Driving assistance package plus)) or via the chassis CAN 1, electronic ignition lock control unit, and chassis CAN 2 to the multifunction camera (with CODE 22P (Lane package)).

Function sequence for Traffic Sign Assist 

At a horizontal angular coverage of ∠ = 46° the multifunction camera records pictures during the journey of the traffic signs over or at the side of the road. Traffic Sign Assist is continuously active in the background. In other words, the corresponding traffic signs are permanently recorded and evaluated by the multifunction camera [MFK]. The recorded picture data are sent to the integrated picture recognition module which compares them against the stored data records and assigns the traffic signs to the respective valid road traffic regulations. The multifunction camera's picture recognition module assigns credibility to all results.

The national regulations for the design and attachment of traffic signs are taken into account for this.

The multifunction camera records all picture objects and evaluates circular as well as square contours. Speed limits, prohibitory and derestriction signs including additional restrictions in countries with signage according to the Vienna convention are accordingly recorded. Non-relevant signs (supplementary signs that, e.g. are relevant for a turnoff lane) are suppressed. If the navigation system is active, supplementary signs will be shown immediately if they are relevant for the currently active navigation.

Possible additional signs include limits in wet conditions or when towing a trailer. If, for example, the "windshield wipers active" status is received by the multifunction camera and a permissible maximum speed with additional restriction "in the wet" is registered, the detected permissible maximum speed is shown in the instrument cluster. The permissible maximum speed is not viewed to be relevant as long as the "windshield wipers active" status is not read in.

Supplementary signs not relevant for passenger cars (e.g. speed limits for trucks) will not be shown. The number of these traffic signs amounts to approx. 10% of all traffic signs. After the data has been evaluated, the detected traffic signs are shown in the multifunction display of the instrument cluster and in the Audio/COMAND display. Information about the currently permissible speed or the currently valid prohibitions is thus available to the driver.

The multifunction camera also combines the information about speed limit to form an event if the information is contradictory.

If the multifunction camera detects a no-entry sign when entering a freeway or expressway in the opposite direction to the direction of travel (wrong-way driver scenario), the "Check direction of travel!" warning message appears on the multifunction display and an acoustic warning is output over the instrument cluster warning buzzer, which is directly actuated by the instrument cluster. (This warning message is only available in Germany).

The multifunction camera (with CODE 22P (Lane package)) sends corresponding requests to display the currently detected traffic sign and a warning message via the chassis CAN 2 to the instrument cluster and via the chassis CAN 2, electronic ignition lock control unit, and interior CAN to the COMAND controller unit.

The stereo multifunction camera (with CODE 23P (Driving assistance package plus)) sends corresponding requests to display the currently detected traffic sign and a warning message via the chassis FlexRay, chassis gateway control unit (with CODE 23P (Driving assistance package plus)), chassis CAN 1, electronic ignition lock control unit, and chassis CAN 2 to the instrument cluster and via the chassis FlexRay, chassis gateway control unit (with CODE 23P (Driving assistance package plus)), chassis CAN 1, electronic ignition lock control unit, and interior CAN to the COMAND controller unit. The COMAND controller unit actuates the Audio/COMAND display via the telematics CAN and sends the corresponding data via the LVDS line.

The acoustic warning message can be acknowledged after t = 5 s at the earliest.

In the event of a fault, the following messages are shown in the multifunction display:

• "System currently not available" (e.g. camera field of view fogged up)
• "System nonfunctional" (e.g. multifunction camera or navigation system malfunction)

Traffic signs recognized by the multifunction camera (example displays) 

Representation of derestriction signs 

The following events cause the derestriction sign to be displayed (picture, position F):

• Passing by a derestriction sign (picture, position F)
• Passing by a derestriction sign with a speed value (picture, position G), if previously no traffic sign with a speed value has been recognized

The following event leads to the display of a derestriction sign (picture, position H):

• Passing a derestriction sign with speed value (picture, position G) which deviates from the speed value of the traffic sign recorded in prior history (picture, position H1)

The following event leads to the display of a derestriction sign (picture, position I):

• Driving by a derestriction sign for no passing (picture, position I) if the no-passing sign was previously detected (picture, position I1)

When driving past a derestriction sign (speed limitation or no passing), the derestriction sign is shown for t = 5 s. The applicable traffic sign is then shown in the multifunction display and in the Audio/COMAND display.

In addition the multifunction camera evaluates information on the area around the vehicle and on the vehicle's own movement.

These include:

• Distance and angle to other road users and objects, detected by the multifunction camera
• Vehicle movement and yaw behavior
• Map data on the COMAND controller unit

The attributes verified in the digital card of the navigation system are also evaluated.

The map data serves to check the object recorded by the multifunction camera for plausibility as well as display speed limits (e.g. 50 km/h) stored in the map when driving into built-up areas.

The map data are used to evaluate the national availability. The national availability may be able to be extended and therefore it matches the current software release.

If the map data cannot be evaluated (e.g. no GPS signals or no map data for the respective country), Traffic Sign Assist is switched to passive after traveling a distance of s = 30 km (the "Traffic Sign Assist nonfunctional" message appears on the multifunction display).

If the multifunction camera detects a "blockage" (e.g. for soiled windshield), the "Traffic Sign Assist not available" message appears. If, later on, a traffic sign is detected again (e.g. reduction in degree of windshield fogging), the "Blocked" mode is deactivated again.

Because speed limits are subject to alteration, the digital map data has to be regularly updated.

Information on the classification of roads and regional development are deduced from the map data provided by COMAND controller unit. By this means the system recognizes that the vehicle is leaving a road for which a speed limit identified earlier was displayed. Since the picture recognition is unable to cancel the display of a speed limit identified outside a town/village when passing a town/village entry sign, use is made of the map data.

With a change from a non-built-up area to a built-up area as defined by the markings for buildings on the navigation map, the system switches to map data. As the town/village limits do not always match the built-up area markings from the navigation map, the display showing the last detected speed limit may disappear too soon or may continue to be shown in the built-up area.

If the multifunction camera detects a speed limitation but this limitation is not repeated (or an additional sign does not appear within a defined distance), the system switches to map data. If no data is referenced in the map, a speed is no longer displayed until a sign is detected again. Situations which lead to a failed cancellation or superfluous cancellation of this display are shown in the following pictures:

Representation of town/village limits and built-up area limits 

Function information and system limits 

Below is a list of example situations in which the pictures recorded by the multifunction camera cannot be evaluated and cannot be used for display in the multifunction display and the Audio/COMAND display:

Speed limits marked by town/village entry and town/village exit signs or by freeway start and finish signs are not recorded. These traffic signs however, are filed through the road definition feature (e.g. freeway) in the digital navigation map.

Electronically actuated changing traffic signs can be "missed" or interpreted incorrectly. In the event of changing traffic signs that are actuated by pulses or flashes, there may be difference to the picture recording's permanent cycle time, depending on the actuated frequency of the changing traffic signs. As a result only fragments of the changing traffic signs will be recognized.

If the image cannot be uniquely identified as a traffic sign, the speed value that is stored in the digital navigation map is then shown.

To enable the detection of changing traffic signs on which the outer circular ring is missing, special detectors have been implemented in the multifunction camera. This type of display is mainly used in the United Kingdom, the Netherlands and Belgium.

The horizontal angle of vision of the multifunction camera is limited. Traffic signs erected immediately after junctions, turnoffs which the vehicle takes, can be "missed" by the multifunction camera.

The horizontal angular coverage of the multifunction camera is ∠ a = 46°.

The vertical angle of vision of the multifunction camera is limited. Traffic signs on sign gantries are only interpreted reliably up to a vehicle speed of v = 160 km/h.

In order to keep recognition certainty homogeneous over all vehicle speed ranges, the evaluation window of the multifunction camera is downsized as the vehicle speed increases. The consequence of limiting the evaluation window of the multifunction camera is that traffic signs installed low down, as used in construction sites, are "missed" if they are passed at high speed.

The resolution of the multifunction camera is limited. Traffic signs on wide or multilane roads can be "missed" as they do not reach the size required for clear identification in the evaluation window of the multifunction camera.

The sensitivity and dynamics of the multifunction camera are limited.

This results in traffic signs not being recognized under the following conditions:

• Traffic signs with low contrast
• Extreme oncoming lights or lights from the rear
• Low levels of lighting, darkness

The focus and alignment of the multifunction camera are fixed. As a result recorded objects are represented with a movement blur depending on the exposure time.

This movement blur and sensitivity of the multifunction camera limit its usefulness at night. The result of the fixed focus and static alignment is that the traffic signs to be recorded must be completely visible in a small observation time window.

Unfavorably designed back-end sections of commercial vehicles may be interpreted as derestriction signs by mistake (left picture). Signage on the back-end section of commercial vehicles may be interpreted as traffic signs by mistake (right picture).

Navigation system data are not available during an update or reset procedure for the COMAND controller unit, which may lead to fault messages occurring in some cases.

Electrical function schematic, traffic sign assist	PE54.30-P-2066-97NBA
Overview of system components for driver assistance systems	GF54.00-P-9998GN