Nvidia upgrades tiny Jetson Orin Nano dev kits for the holidays
(2024/12/17)
- Reference: 1734465612
- News link: https://www.theregister.co.uk/2024/12/17/nvidia_jetson_orin/
- Source link:
Nvidia is bringing the AI hype home for the holidays with the launch of a tiny new dev board called the Jetson Orin Nano Super.
Nvidia cheekily bills [1]the board as the "most affordable generative AI supercomputer," though that might be stretching the term quite a bit.
The dev kit consists of a system on module, similar to a Raspberry Pi Compute Module, that sits atop a reference carrier board for I/O and power. And, just like a Raspberry Pi, Nvidia's diminutive dev kit is aimed at developers and hobbyists looking to experiment with generative AI.
[2]
Similar to a Raspberry Pi Compute Module, Nvidia's Jetson Orin Nano development kit consists of a compute module containing the SoC that attaches to a carrier board for I/O and power. - Click to enlarge
Under the hood, the Orin Nano features six Arm Cortex-A78AE cores along with an Nvidia GPU based on its older Ampere architecture with 1024 CUDA cores and 32 tensor cores.
The design appears to be identical to the [3]original Jetson Orin Nano. However, Nvidia says the "Super" edition of the board is both faster and less expensive, coming in at $249 versus the original $499 price tag.
[4]
In terms of performance, the Jetson Orin Nano Super packs 67 TOPS at INT8, which is faster than the NPUs in any of Intel, AMD, or Qualcomm's latest AI PCs backed by 8GB of LPDDR5 memory capable of 102GB/s of memory bandwidth. According to Nvidia, these specs reflect a 70 percent uplift in performance and 50 percent more memory bandwidth than its predecessor.
[5]
[6]
The bandwidth boost is particularly important for those looking to play with the kind of large language models (LLMs) that power modern AI chatbots at home. At 102GB/s, we estimate the dev kit should be able to generate words at around 18-20 tokens a second when running a 4-bit quantized version of Meta's 8-billion-parameter Llama 3.1 model.
If you're curious about how TOPS, memory capacity, and bandwidth relate to model performance, you can check out our guide [7]here .
[8]
For I/O, the dev kit's carrier board features the usual fare of connectivity for an SBC, including gigabit Ethernet, DisplayPort, four USB 3.2 Gen 2 type-A ports, USB-C, dual M.2 slots with M and E keys, along with a variety of expansion headers.
[9]Buying a PC for local AI? These are the specs that actually matter
[10]Just how deep is Nvidia's CUDA moat really?
[11]Take a closer look at Nvidia's buy of Run.ai, European Commission told
[12]Cheat codes for LLM performance: An introduction to speculative decoding
In terms of software support, you might think those Arm cores might be problematic; however, that really isn't the case. Nvidia has supported GPUs on Arm processors for years, with its most sophisticated [13]designs - the GH200 and GB200 - utilizing its custom Arm-based Grace CPU. This means you can expect broad support for the GPU giant's software suite including Nvidia Isaac, Metropolis, and Holoscan, to name a few.
Along with a variety of open AI models available via Nvidia's Jetson AI lab, the dev kit also supports up to four cameras for robotics or vision processing workloads.
Alongside the new dev kit, Nvidia is also rolling out a software update to its older Jetson Orin NX and Nano system on modules, which it says should boost GenAI performance by 1.7x, so if you already picked up an Orin Nano, you shouldn't be missing out on too much. ®
Get our [14]Tech Resources
[1] https://blogs.nvidia.com/blog/jetson-generative-ai-supercomputer/
[2] https://regmedia.co.uk/2024/12/17/nvidia_jetson_orin_dev_kit.jpg
[3] https://www.theregister.com/2022/09/20/nvidia_robots_auto_omniverse/
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/systems&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2Z2ICkdJudNbAEDmQc2xDDgAAAAY&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/systems&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Z2ICkdJudNbAEDmQc2xDDgAAAAY&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/systems&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Z2ICkdJudNbAEDmQc2xDDgAAAAY&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[7] https://www.theregister.com/2024/08/25/ai_pc_buying_guide/
[8] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/systems&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Z2ICkdJudNbAEDmQc2xDDgAAAAY&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[9] https://www.theregister.com/2024/08/25/ai_pc_buying_guide/
[10] https://www.theregister.com/2024/12/17/nvidia_cuda_moat/
[11] https://www.theregister.com/2024/12/16/probe_nvidias_buy_of_runai/
[12] https://www.theregister.com/2024/12/15/speculative_decoding/
[13] https://www.theregister.com/2024/03/18/nvidia_turns_up_the_ai/?td=rt-9c
[14] https://whitepapers.theregister.com/
Nvidia cheekily bills [1]the board as the "most affordable generative AI supercomputer," though that might be stretching the term quite a bit.
The dev kit consists of a system on module, similar to a Raspberry Pi Compute Module, that sits atop a reference carrier board for I/O and power. And, just like a Raspberry Pi, Nvidia's diminutive dev kit is aimed at developers and hobbyists looking to experiment with generative AI.
[2]
Similar to a Raspberry Pi Compute Module, Nvidia's Jetson Orin Nano development kit consists of a compute module containing the SoC that attaches to a carrier board for I/O and power. - Click to enlarge
Under the hood, the Orin Nano features six Arm Cortex-A78AE cores along with an Nvidia GPU based on its older Ampere architecture with 1024 CUDA cores and 32 tensor cores.
The design appears to be identical to the [3]original Jetson Orin Nano. However, Nvidia says the "Super" edition of the board is both faster and less expensive, coming in at $249 versus the original $499 price tag.
[4]
In terms of performance, the Jetson Orin Nano Super packs 67 TOPS at INT8, which is faster than the NPUs in any of Intel, AMD, or Qualcomm's latest AI PCs backed by 8GB of LPDDR5 memory capable of 102GB/s of memory bandwidth. According to Nvidia, these specs reflect a 70 percent uplift in performance and 50 percent more memory bandwidth than its predecessor.
[5]
[6]
The bandwidth boost is particularly important for those looking to play with the kind of large language models (LLMs) that power modern AI chatbots at home. At 102GB/s, we estimate the dev kit should be able to generate words at around 18-20 tokens a second when running a 4-bit quantized version of Meta's 8-billion-parameter Llama 3.1 model.
If you're curious about how TOPS, memory capacity, and bandwidth relate to model performance, you can check out our guide [7]here .
[8]
For I/O, the dev kit's carrier board features the usual fare of connectivity for an SBC, including gigabit Ethernet, DisplayPort, four USB 3.2 Gen 2 type-A ports, USB-C, dual M.2 slots with M and E keys, along with a variety of expansion headers.
[9]Buying a PC for local AI? These are the specs that actually matter
[10]Just how deep is Nvidia's CUDA moat really?
[11]Take a closer look at Nvidia's buy of Run.ai, European Commission told
[12]Cheat codes for LLM performance: An introduction to speculative decoding
In terms of software support, you might think those Arm cores might be problematic; however, that really isn't the case. Nvidia has supported GPUs on Arm processors for years, with its most sophisticated [13]designs - the GH200 and GB200 - utilizing its custom Arm-based Grace CPU. This means you can expect broad support for the GPU giant's software suite including Nvidia Isaac, Metropolis, and Holoscan, to name a few.
Along with a variety of open AI models available via Nvidia's Jetson AI lab, the dev kit also supports up to four cameras for robotics or vision processing workloads.
Alongside the new dev kit, Nvidia is also rolling out a software update to its older Jetson Orin NX and Nano system on modules, which it says should boost GenAI performance by 1.7x, so if you already picked up an Orin Nano, you shouldn't be missing out on too much. ®
Get our [14]Tech Resources
[1] https://blogs.nvidia.com/blog/jetson-generative-ai-supercomputer/
[2] https://regmedia.co.uk/2024/12/17/nvidia_jetson_orin_dev_kit.jpg
[3] https://www.theregister.com/2022/09/20/nvidia_robots_auto_omniverse/
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/systems&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2Z2ICkdJudNbAEDmQc2xDDgAAAAY&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/systems&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Z2ICkdJudNbAEDmQc2xDDgAAAAY&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/systems&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Z2ICkdJudNbAEDmQc2xDDgAAAAY&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[7] https://www.theregister.com/2024/08/25/ai_pc_buying_guide/
[8] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/systems&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Z2ICkdJudNbAEDmQc2xDDgAAAAY&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[9] https://www.theregister.com/2024/08/25/ai_pc_buying_guide/
[10] https://www.theregister.com/2024/12/17/nvidia_cuda_moat/
[11] https://www.theregister.com/2024/12/16/probe_nvidias_buy_of_runai/
[12] https://www.theregister.com/2024/12/15/speculative_decoding/
[13] https://www.theregister.com/2024/03/18/nvidia_turns_up_the_ai/?td=rt-9c
[14] https://whitepapers.theregister.com/
Re: specs
Empire of the Pussycat
'When I click on "Purchase Now" ... I get "This site can’t be reached"'
It's probably an AI hallucination.
Re: specs
David 132
Interesting. Works for me - I (in the US, FWIW) get links for purchasing it from Seeed, Arrow, and Sparkfun.
Here's the links in case you're still having issues.
Note that as I write this, all 3 distis report it as "out of stock".
Seeed: https://www.seeedstudio.com/NVIDIAr-Jetson-Orintm-Nano-Developer-Kit-p-5617.html?nvid=em-945-13766-0005-000
Sparkfun: https://www.sparkfun.com/products/22098?nvid=em-945-13766-0005-000
Arrow: https://www.arrow.com/en/products/945-13766-0000-000/nvidia?nvid=em-945-13766-0005-000
specs
From the specs:
NVIDIA Ampere architecture with 1024 CUDA cores and 32 tensor cores
8GB 128-bit LPDDR5 102GB/s
When I click on "Purchase Now" ... I get "This site can’t be reached"