{"id":17275,"date":"2026-09-05T14:22:17","date_gmt":"2026-09-05T12:22:17","guid":{"rendered":"https:\/\/www.car-shooters.com\/en\/2026\/09\/05\/sense-how-a-self-driving-car-actually-sees-the-world\/"},"modified":"2026-09-05T14:22:17","modified_gmt":"2026-09-05T12:22:17","slug":"sense-how-a-self-driving-car-actually-sees-the-world","status":"publish","type":"post","link":"https:\/\/www.car-shooters.com\/en\/2026\/09\/05\/sense-how-a-self-driving-car-actually-sees-the-world\/","title":{"rendered":"Sense: How a Self-Driving Car Actually Sees the World"},"content":{"rendered":"

We have already explained the SAE levels of autonomy<\/strong><\/a> — the framework for who’s legally driving. What we haven’t explained is how the car itself actually pulls off a Level 4 drive, and that starts with the first of three stages every automated vehicle runs through, thousands of times a second: sense, think, act<\/strong>. This article covers the first one. Before a car can decide anything, it has to build an accurate, continuously updated model of everything around it — and that turns out to be a far harder problem than strapping on a few cameras.<\/p>\n

 <\/p>\n

Three sensor types, three different ways to be wrong<\/h2>\n

No single sensor technology is good enough on its own, which is precisely why every serious autonomous driving system carries several. Cameras<\/strong> are cheap and see the one thing nothing else can — colour, text, lane markings, traffic light state — but they degrade exactly when driving gets hardest: low sun glare, heavy rain, fog, or night-time all directly hurt the image a camera can extract useful information from. Radar<\/strong> solves the opposite problem: radio waves shrug off rain, fog and darkness, and directly measure a target’s velocity via Doppler shift, which is why radar has anchored automatic emergency braking and adaptive cruise control for two decades — but its spatial resolution is coarse, it struggles to classify what it’s looking at, and it can’t read a sign or a light. LiDAR<\/strong> spins a laser around the car thousands of times a second and times the reflections to build a precise 3D point cloud of the environment, giving centimetre-accurate distance and shape data neither camera nor radar can match — at the cost of being the most expensive sensor on the car, and one whose own performance drops in heavy snow, fog or dust, when laser pulses scatter off particles in the air instead of the target beyond them.<\/p>\n

 <\/p>\n

Fusion: turning three flawed pictures into one trustworthy one<\/h2>\n

Because each sensor fails in different conditions, the practical answer isn’t picking a winner — it’s sensor fusion<\/strong>: combining camera, radar and LiDAR data into a single, more reliable model of the world than any one sensor could produce alone. Fusion can happen at different points in the pipeline: low-level (raw) fusion<\/strong> combines unprocessed sensor signals before individual objects are even identified, preserving the most information but demanding far more computing power; object-level (late) fusion<\/strong> lets each sensor independently detect objects first, then merges those separate detections, which is computationally lighter but can lose detail one sensor alone would have caught. Either way, the point of fusion is redundancy with a purpose: if a camera is blinded by sun glare on a target radar has already flagged and tracked by velocity, the car doesn’t lose that object — it simply trusts the sensor that’s still working.<\/p>\n

 <\/p>\n

The hardest problem: sensing what you can’t see at all<\/h2>\n

Every sensor described so far shares one fundamental limit: it can only detect what’s in its direct line of sight. A pedestrian stepping out from behind a parked delivery van, or a cyclist hidden by the car ahead at a junction, is invisible to cameras, radar and LiDAR alike until the moment it’s already too late to react comfortably — this is occlusion<\/strong>, and it’s one of the last genuinely unsolved problems in autonomous perception, precisely because no amount of better hardware on the car itself can see around a solid object in the way.<\/p>\n

 <\/p>\n

Where the state of the art is heading: EU research on perception<\/h2>\n

Making onboard sensing genuinely robust has been the subject of one of the largest automotive electronics research efforts the EU has ever funded. PRYSTINE<\/a><\/strong> (2018-2021, €14.37M, coordinated by Infineon Technologies in Germany, under the ECSEL Joint Undertaking) set out to achieve what it called “Fail-operational Urban SuRround PercepTION” — robust radar and LiDAR sensor fusion married to control functions specifically engineered to keep working, not just detect correctly, when conditions or components degrade. Fusion hardware itself is still evolving: PARALIA<\/a><\/strong> (2023-2027, €5.00M, coordinated by Aristotle University of Thessaloniki, Greece) is re-architecting the sensors themselves, developing a shared photonic radar\/LiDAR beamforming platform aimed at ultra-high resolution at ultra-long range, at a fraction of today’s cost and power draw. And HIDDEN<\/a><\/strong> (2025-2028, €5.00M, coordinated by ICCS, the Institute of Communication and Computer Systems in Greece) goes after the occlusion problem specifically, developing collective awareness<\/strong> techniques that let a vehicle detect occluded objects and vulnerable road users with or without help from road infrastructure, predicting their short-term movement using behavioural models, and combining that machine-generated picture with driver gaze-tracking in a hybrid human-machine intelligence loop.<\/p>\n

 <\/p>\n

Why it matters beyond the spec sheet<\/h2>\n

A self-driving car’s decisions can only ever be as good as the picture of the world it’s deciding from — which is why perception, not the eventual steering command, is where an autonomous system is most likely to fail first. Once that picture exists, though, having it is only half the job: turning it into an actual driving decision is a problem of its own, and that’s where we go next.<\/p>\n

 <\/p>\n

Photo:<\/b> © Car-Shooters<\/i><\/p>\n","protected":false},"excerpt":{"rendered":"

We have already explained the SAE levels of autonomy — the framework for who’s legally driving. What we haven’t explained is how the car itself actually pulls off a Level 4 drive, and that starts with the first of three stages every automated vehicle runs through, thousands of times a second: sense, think, act. This … Continued<\/a><\/p>\n","protected":false},"author":1,"featured_media":16214,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_yoast_wpseo_focuskw":"self driving car sensors","_yoast_wpseo_metadesc":"Every autonomous driving decision starts with a picture of the world built from cameras, radar and LiDAR data that individually can each be fooled. Here's how a self-driving car actually perceives its surroundings, and why the hardest problem is the object it can't see at all.","footnotes":""},"categories":[285],"tags":[],"class_list":["post-17275","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-focus-en"],"yoast_head":"\nSense: How a Self-Driving Car Actually Sees the World | Car - Shooters<\/title>\n<meta name=\"description\" content=\"Every autonomous driving decision starts with a picture of the world built from cameras, radar and LiDAR data that individually can each be fooled. Here's how a self-driving car actually perceives its surroundings, and why the hardest problem is the object it can't see at all.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.car-shooters.com\/en\/2026\/09\/05\/sense-how-a-self-driving-car-actually-sees-the-world\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Sense: How a Self-Driving Car Actually Sees the World | Car - Shooters\" \/>\n<meta property=\"og:description\" content=\"Every autonomous driving decision starts with a picture of the world built from cameras, radar and LiDAR data that individually can each be fooled. Here's how a self-driving car actually perceives its surroundings, and why the hardest problem is the object it can't see at all.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.car-shooters.com\/en\/2026\/09\/05\/sense-how-a-self-driving-car-actually-sees-the-world\/\" \/>\n<meta property=\"og:site_name\" content=\"Car - Shooters\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/carshooters\" \/>\n<meta property=\"article:published_time\" content=\"2026-09-05T12:22:17+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.car-shooters.com\/wp-content\/uploads\/2026\/02\/CAR-SHOOTERS-Tesla-Model-3-Performance-Prova-su-Strada-20.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1920\" \/>\n\t<meta property=\"og:image:height\" content=\"1278\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Edoardo Mascalchi\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@Car_Shooters\" \/>\n<meta name=\"twitter:site\" content=\"@Car_Shooters\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Edoardo Mascalchi\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.car-shooters.com\/en\/2026\/09\/05\/sense-how-a-self-driving-car-actually-sees-the-world\/\",\"url\":\"https:\/\/www.car-shooters.com\/en\/2026\/09\/05\/sense-how-a-self-driving-car-actually-sees-the-world\/\",\"name\":\"Sense: How a Self-Driving Car Actually Sees the World | Car - Shooters\",\"isPartOf\":{\"@id\":\"https:\/\/www.car-shooters.com\/en\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.car-shooters.com\/en\/2026\/09\/05\/sense-how-a-self-driving-car-actually-sees-the-world\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.car-shooters.com\/en\/2026\/09\/05\/sense-how-a-self-driving-car-actually-sees-the-world\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.car-shooters.com\/wp-content\/uploads\/2026\/02\/CAR-SHOOTERS-Tesla-Model-3-Performance-Prova-su-Strada-20.jpg\",\"datePublished\":\"2026-09-05T12:22:17+00:00\",\"author\":{\"@id\":\"https:\/\/www.car-shooters.com\/en\/#\/schema\/person\/724c222d4f54a294e9a3c9215b18513e\"},\"description\":\"Every autonomous driving decision starts with a picture of the world built from cameras, radar and LiDAR data that individually can each be fooled. Here's how a self-driving car actually perceives its surroundings, and why the hardest problem is the object it can't see at all.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.car-shooters.com\/en\/2026\/09\/05\/sense-how-a-self-driving-car-actually-sees-the-world\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.car-shooters.com\/en\/2026\/09\/05\/sense-how-a-self-driving-car-actually-sees-the-world\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.car-shooters.com\/en\/2026\/09\/05\/sense-how-a-self-driving-car-actually-sees-the-world\/#primaryimage\",\"url\":\"https:\/\/www.car-shooters.com\/wp-content\/uploads\/2026\/02\/CAR-SHOOTERS-Tesla-Model-3-Performance-Prova-su-Strada-20.jpg\",\"contentUrl\":\"https:\/\/www.car-shooters.com\/wp-content\/uploads\/2026\/02\/CAR-SHOOTERS-Tesla-Model-3-Performance-Prova-su-Strada-20.jpg\",\"width\":1920,\"height\":1278,\"caption\":\"Tesla Model 3 Performance \u2013 detail, headlight and front fascia close-up\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.car-shooters.com\/en\/2026\/09\/05\/sense-how-a-self-driving-car-actually-sees-the-world\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.car-shooters.com\/en\/home\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Sense: How a Self-Driving Car Actually Sees the World\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.car-shooters.com\/en\/#website\",\"url\":\"https:\/\/www.car-shooters.com\/en\/\",\"name\":\"Car - Shooters\",\"description\":\"Beautiful cars in fantastic places\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.car-shooters.com\/en\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.car-shooters.com\/en\/#\/schema\/person\/724c222d4f54a294e9a3c9215b18513e\",\"name\":\"Edoardo Mascalchi\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.car-shooters.com\/en\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/630e3e904571fd05cbd5e16ba5d77db9?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/630e3e904571fd05cbd5e16ba5d77db9?s=96&d=mm&r=g\",\"caption\":\"Edoardo Mascalchi\"},\"sameAs\":[\"http:\/\/www.edoardomascalchi.it\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Sense: How a Self-Driving Car Actually Sees the World | Car - Shooters","description":"Every autonomous driving decision starts with a picture of the world built from cameras, radar and LiDAR data that individually can each be fooled. Here's how a self-driving car actually perceives its surroundings, and why the hardest problem is the object it can't see at all.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.car-shooters.com\/en\/2026\/09\/05\/sense-how-a-self-driving-car-actually-sees-the-world\/","og_locale":"en_US","og_type":"article","og_title":"Sense: How a Self-Driving Car Actually Sees the World | Car - Shooters","og_description":"Every autonomous driving decision starts with a picture of the world built from cameras, radar and LiDAR data that individually can each be fooled. Here's how a self-driving car actually perceives its surroundings, and why the hardest problem is the object it can't see at all.","og_url":"https:\/\/www.car-shooters.com\/en\/2026\/09\/05\/sense-how-a-self-driving-car-actually-sees-the-world\/","og_site_name":"Car - Shooters","article_publisher":"https:\/\/www.facebook.com\/carshooters","article_published_time":"2026-09-05T12:22:17+00:00","og_image":[{"width":1920,"height":1278,"url":"https:\/\/www.car-shooters.com\/wp-content\/uploads\/2026\/02\/CAR-SHOOTERS-Tesla-Model-3-Performance-Prova-su-Strada-20.jpg","type":"image\/jpeg"}],"author":"Edoardo Mascalchi","twitter_card":"summary_large_image","twitter_creator":"@Car_Shooters","twitter_site":"@Car_Shooters","twitter_misc":{"Written by":"Edoardo Mascalchi","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.car-shooters.com\/en\/2026\/09\/05\/sense-how-a-self-driving-car-actually-sees-the-world\/","url":"https:\/\/www.car-shooters.com\/en\/2026\/09\/05\/sense-how-a-self-driving-car-actually-sees-the-world\/","name":"Sense: How a Self-Driving Car Actually Sees the World | Car - Shooters","isPartOf":{"@id":"https:\/\/www.car-shooters.com\/en\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.car-shooters.com\/en\/2026\/09\/05\/sense-how-a-self-driving-car-actually-sees-the-world\/#primaryimage"},"image":{"@id":"https:\/\/www.car-shooters.com\/en\/2026\/09\/05\/sense-how-a-self-driving-car-actually-sees-the-world\/#primaryimage"},"thumbnailUrl":"https:\/\/www.car-shooters.com\/wp-content\/uploads\/2026\/02\/CAR-SHOOTERS-Tesla-Model-3-Performance-Prova-su-Strada-20.jpg","datePublished":"2026-09-05T12:22:17+00:00","author":{"@id":"https:\/\/www.car-shooters.com\/en\/#\/schema\/person\/724c222d4f54a294e9a3c9215b18513e"},"description":"Every autonomous driving decision starts with a picture of the world built from cameras, radar and LiDAR data that individually can each be fooled. Here's how a self-driving car actually perceives its surroundings, and why the hardest problem is the object it can't see at all.","breadcrumb":{"@id":"https:\/\/www.car-shooters.com\/en\/2026\/09\/05\/sense-how-a-self-driving-car-actually-sees-the-world\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.car-shooters.com\/en\/2026\/09\/05\/sense-how-a-self-driving-car-actually-sees-the-world\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.car-shooters.com\/en\/2026\/09\/05\/sense-how-a-self-driving-car-actually-sees-the-world\/#primaryimage","url":"https:\/\/www.car-shooters.com\/wp-content\/uploads\/2026\/02\/CAR-SHOOTERS-Tesla-Model-3-Performance-Prova-su-Strada-20.jpg","contentUrl":"https:\/\/www.car-shooters.com\/wp-content\/uploads\/2026\/02\/CAR-SHOOTERS-Tesla-Model-3-Performance-Prova-su-Strada-20.jpg","width":1920,"height":1278,"caption":"Tesla Model 3 Performance \u2013 detail, headlight and front fascia close-up"},{"@type":"BreadcrumbList","@id":"https:\/\/www.car-shooters.com\/en\/2026\/09\/05\/sense-how-a-self-driving-car-actually-sees-the-world\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.car-shooters.com\/en\/home\/"},{"@type":"ListItem","position":2,"name":"Sense: How a Self-Driving Car Actually Sees the World"}]},{"@type":"WebSite","@id":"https:\/\/www.car-shooters.com\/en\/#website","url":"https:\/\/www.car-shooters.com\/en\/","name":"Car - Shooters","description":"Beautiful cars in fantastic places","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.car-shooters.com\/en\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.car-shooters.com\/en\/#\/schema\/person\/724c222d4f54a294e9a3c9215b18513e","name":"Edoardo Mascalchi","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.car-shooters.com\/en\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/630e3e904571fd05cbd5e16ba5d77db9?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/630e3e904571fd05cbd5e16ba5d77db9?s=96&d=mm&r=g","caption":"Edoardo Mascalchi"},"sameAs":["http:\/\/www.edoardomascalchi.it"]}]}},"translations":{"fr":{"id":17276,"slug":"sense-comment-une-voiture-autonome-percoit-vraiment-le-monde","link":"https:\/\/www.car-shooters.com\/fr\/2026\/09\/05\/sense-comment-une-voiture-autonome-percoit-vraiment-le-monde\/"},"it":{"id":17274,"slug":"sense-come-vede-davvero-il-mondo-unauto-a-guida-autonoma","link":"https:\/\/www.car-shooters.com\/2026\/09\/05\/sense-come-vede-davvero-il-mondo-unauto-a-guida-autonoma\/"}},"_links":{"self":[{"href":"https:\/\/www.car-shooters.com\/en\/wp-json\/wp\/v2\/posts\/17275","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.car-shooters.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.car-shooters.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.car-shooters.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.car-shooters.com\/en\/wp-json\/wp\/v2\/comments?post=17275"}],"version-history":[{"count":0,"href":"https:\/\/www.car-shooters.com\/en\/wp-json\/wp\/v2\/posts\/17275\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.car-shooters.com\/en\/wp-json\/wp\/v2\/media\/16214"}],"wp:attachment":[{"href":"https:\/\/www.car-shooters.com\/en\/wp-json\/wp\/v2\/media?parent=17275"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.car-shooters.com\/en\/wp-json\/wp\/v2\/categories?post=17275"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.car-shooters.com\/en\/wp-json\/wp\/v2\/tags?post=17275"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}