What we know as of today for HW3 (AP3), Teslas NN chip and V9 and above
2018 – Partner no more – Mobileye and Nvidia dumped as suppliers of autopilot ( the only sure information )
Tesla dumped Mobileye (EyeQ3). Mobileye is working with BMW and Intel and has been a good flirt with regulators, but apparently was not moving fast enough, with current estimate of L4 set for 2021.
Looks like Tesla dumped Nvidia as well. The Nvidia PX2 was rumored to cost up to $15000 per unit, with bulk prices eventually targeted to cost in the 2000 dollars range. Outside of the cost of retroffiting all the Tesla the vehicles sold on the promise of potential full autonomy, one benefit of having Tesla developed /specced chips would be better power consumption, always a consideration in electric vehicles.
2019 – early to mid – planned delivery of Tesla HW3 with Teslas own NN chip
Musk tweets in 2018 announced the chip is improving Tesla Autopilot performance between 500% and 2000%.
Version 9 /HW3 – everything below is gossip only
- Camera agnostic. At the limit of what the before HW3 hardware could provide. For more info, see this forum post from 2018 by Jimmy_d
- Depth through stereo+ – using all cameras allows for 360 vision, better depth perception and thus better object recognition
- One network handles all 8 cameras
- Same weight file being used for all cameras (this has pretty interesting implications and previously V8 main/narrow seems to have had separate weights for each camera)
- Processed resolution of 3 front cameras and back camera: 1280×960 (full camera resolution)
- Processed resolution of pillar and repeater cameras: 640×480 (1/2×1/2 of camera’s true resolution)
- all cameras: 3 color channels, 2 frames (2 frames also has very interesting implications) (was 640×416, 2 color channels, 1 frame, only main and narrow in V8)
“This V9 network is a monster, and that’s not the half of it. When you increase the number of parameters (weights) in an NN by a factor of 5 you don’t just get 5 times the capacity and need 5 times as much training data. In terms of expressive capacity increase it’s more akin to a number with 5 times as many digits. So if V8’s expressive capacity was 10, V9’s capacity is more like 100,000. It’s a mind boggling expansion of raw capacity. And likewise the amount of training data doesn’t go up by a mere 5x. It probably takes at least thousands and perhaps millions of times more data to fully utilize a network that has 5x as many parameters.
This network is far larger than any vision NN I’ve seen publicly disclosed and I’m just reeling at the thought of how much data it must take to train it. I sat on this estimate for a long time because I thought that I must have made a mistake. But going over it again and again I find that it’s not my calculations that were off, it’s my expectations that were off.”
“Scaling computational power, training data, and industrial resources plays to Tesla’s strengths and involves less uncertainty than potentially more powerful but less mature techniques. At the same time Tesla is doubling down on their ‘vision first / all neural networks’ approach and, as far as I can tell, it seems to be going well.”
Tesla plans for hw3 – release mid 2019
- retrofit in vehicles with existing 2.0 and 2.5 versions for those who buy Full Self-Driving Capability Package when its released.
Who / what is making the chip and the pilot hardware?
Tesla likes making everything in-house. Even the seats. So what is happening with the NN chip and the pilot hardware?
We know Tesla poached top people from AMD. We know samsung is supplying something. But what is it and who is going to make it?
Q3 Earnings call, October 2018, Pete Bannon (former chip designer at Apple, now Teslas engineering head) transcript
“Hi, this is Pete Bannon. The Hardware 3 design is continuing to move along. Over the last quarter, we’ve completed qualification of the silicon, qualification of the board. We started the manufacturing line, qualification of the manufacturing line. We’ve been validating the provisioning flows in the factory. We built test versions of Model S, X and 3 in the factory to validate all the fit and finish of the parts and all the provisioning flows.”
Q3 statement from Andrew Karpathy
“we are currently at a place where we trained large neural networks that work very well, but we are not able to deploy them to the fleet due to computational constraints. So, all of this will change with the next iteration of the hardware. And it’s a massive step improvement in the compute capability. And the team is incredibly excited to get these networks out there.”
What we know today – rumors
from this reddit thread
- Samsung Exynos 7xxx SoC, based on the existence of ARM A72 cores (this would not be a super new SoC, as the Exynos SoC is about an Oct 2015 vintage). HW3 CPU cores are clocked at 1.6GHz, with a MALI GPU at 250MHz and memory speed 533MHz.
- HW3 architecture is similar to HW2.5 in that there are two separate compute nodes (called “sides”): the “A” side that does all the work and the “B” side that currently does not do anything.
- Also, it appears there are some devices attached to this SoC. Obviously, there is some emmc storage, but more importantly there’s a Tesla PCI-Ex device named “TRIP” that works as the NN accelerator. The name might be an acronym for “Tensor <something> Inference Processor”. In fact, there are at least two such “TRIP” devices, and maybe possibly two per “side”.
Autonomous 