News: 1692865870

  ARM Give a man a fire and he's warm for a day, but set fire to him and he's warm for the rest of his life (Terry Pratchett, Jingo)

Why these cloud-connected 3D printers started making junk all by themselves

(2023/08/24)


On August 15, 2023, 3D printer maker Bambu Lab experienced a service outage caused by what it’s described as "abnormal network traffic" that caused some customers' cloud-connected printers to start creating plastic objects without user intervention.

Following the phantom printing, some customers complained on social media that their printers, which cost between $599 and $1,199 (£475 to £950), had [1]been [2]damaged . In some cases, printers tried to build objects on printer plates that were already occupied.

In an [3]incident review posted last week, the manufacturer – [4]based in China with a US division in Austin, Texas – blamed the snafu on the [5]MQTT SDK that relays network messages between the printers and its cloud services.

[6]

"One of the two MQTT SDK clients was disconnected due to a timeout," the firm explained. "Usually, the service will automatically restore the connection, but in this case, the service returned a successful connection report when, in reality, the connection was not re-established. During this time, a number of print start messages accumulated."

[7]

[8]

There was another connection failure, after which Bambu's cloud service managed to re-establish communication – but in doing so, it dumped the backed-up print start messages to "a number of printers" and began printing if the printer was idle.

Further complicating matters, an API couldn't handle the large number of access requests that occurred during the cloud service interruption. The Bambu Studio app, which handles print tasks, is programmed to retry requests after accessing the cloud. Those requests further burdened the API service and exceeded its capacity.

[9]Don't believe the hype: HP CEO says 3D printing hasn't met early hopes

[10]Cloud snafu leads to 3D printer user printing on a stranger's kit

[11]How to make a stranger's insecure 3D printer halt-and-catch-fire

[12]The answer to 3D printing equipment on Mars might lie in the Red Planet's dust

To prevent further print queue dumps, Bambu Lab has already revised its service logic to increase database connection sizes – a change that should improve throughput. The outfit also plans to make changes to its printer firmware and cloud printing logic.

Firmware for the company’s X1 Series printers, which come equipped with LIDAR sensors, will gain functionality that checks to see whether models have been removed from the printing plate. This should stop the machines creating objects on printing plates that are occupied.

[13]

Bambu’s P1 Series, which lacks LIDAR, will be updated to prompt users to clean the plate before starting a print job.

These features will be enabled by default but can be disabled. That's similar to the networking setup in the Bambu Studio application – it [14]defaults to cloud connectivity but can be switched for LAN-based operation.

Bambu’s cloud printing logic is also slated to see further refinement. When printing is initiated, the printer will check the timestamp and will toss print jobs that fall outside a specified time window. Hopefully this will avoid massive backlogs building up.

[15]

In a [16]note published during its incident investigation, Bambu said it plans to improve LAN Mode for its printers. That change will allow the devices to be used during a future cloud outages. The note also points out that printing from SD cards is already available as an entirely offline option.

Customers had already [17]requested the ability to print without connecting to the manufacturer's cloud service before this event.

Turning up the heat

The 3D printer biz also plans to monitor the temperature of printer components like the "hotend" – which melts the filament so it can be extruded.

"If a fault is detected, an error message will be prompted on the printer screen, Bambu Studio, and Bambu Handy, to alert the user and avoid any potential hazards," the org announced. "Of course, the heaters will be turned off to further minimize any potential risk."

In January, Bambu insisted that the "hotend" in its printers "can only reach a maximum temperature of around 400°C [752°F]," a temperature it assured is too low [18]to start a fire .

Bambu Lab says that customers who experienced hardware failure as a result of the outage will get help with printer repair – in terms of replacement parts and spools of filament.

Additionally, two spools of randomly selected PLA filament (which retails around $25–$50) will be provided "as compensation for the trouble this has caused."

Bambu Lab promises that the firmware changes mentioned above are coming soon. Indeed, some were planned before this SNAFU. Owners should not wait up late in the hope of imminent arrival: Bambu has advised that "to ensure the quality and reliability of software updates and to also be able to bring you other new features we are working on, we will need some time to do all the changes."

The company has not enumerated "some time" and at the time of writing had not responded to a request for comment. ®

Get our [19]Tech Resources



[1] https://www.reddit.com/r/BambuLab/comments/15x2cl8/are_bambu_replacing_the_damaged_parts/

[2] https://www.reddit.com/r/BambuLab/comments/15rznpy/printer_started_printing_by_itself_damaged_itself/

[3] https://blog.bambulab.com/update-for-cloud-downtime/

[4] https://bambulab.com/en/policies/terms

[5] https://mqtt.org/

[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/devops&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZOcqQmaVQCMDVQoSxcfbfgAAA9I&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0

[7] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/devops&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZOcqQmaVQCMDVQoSxcfbfgAAA9I&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0

[8] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/devops&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZOcqQmaVQCMDVQoSxcfbfgAAA9I&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0

[9] https://www.theregister.com/2022/10/24/hp_lores_3d_printing_not_met_hype/

[10] https://www.theregister.com/2021/08/20/3d_printer_spaghetti_detectives_cloud_config/

[11] https://www.theregister.com/2020/04/13/security_roundup_100420/

[12] https://www.theregister.com/2022/09/07/mars_dust_3d_printing/

[13] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/devops&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZOcqQmaVQCMDVQoSxcfbfgAAA9I&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0

[14] https://wiki.bambulab.com/en/software/bambu-studio/remote-control

[15] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/devops&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZOcqQmaVQCMDVQoSxcfbfgAAA9I&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0

[16] https://blog.bambulab.com/cloud-temporary-outage-investigation/

[17] https://forum.bambulab.com/t/petition-for-fully-functional-lan-mode-please-like-if-you-agree/8211

[18] https://blog.bambulab.com/thermal-runaway/

[19] https://whitepapers.theregister.com/



Filippo

I assume that, at least, they were their own print jobs, and the problem was that they were started out-of-queue? From the article, it seems like the failed print jobs were sent through the cloud to random customers' printers. That would be... even more interesting.

Sounds like this cloud thing was programmed as if it was a local server

Pascal Monett

We all know that cloud = someone else's server.

In this case, it would seem that someone else's server had an issue with restoring a proper job list after a connection had dropped (sounds like something that should have been tested properly).

That looks like there isn't sufficient job identification when printing. If the printer knew which job it was printing and if each new instruction was accompanied by the proper jobID, then if a new jobID was suddenly sending instructions, the printer would be able to refuse and set itself in error status.

Sounds like that is a precaution that was not taken because why think of making sure the printer knows what it is printing ? The Cloud TM never goes wrong, right ?

Stuart Castle

Why does a 3d printer need to be cloud controlled? That would seem to be over complicating things massively.

The cloud does offer the advantage that the device is accessible wherever the user happens to be in the world, but with any 3D printer, you need to ensure someone is there to remove the print from the plate.

The only advantage I can think of is if you are out, you can trigger a print job (these can be quite lengthy), so it should be ready by the time you get to the printer.

b0llchit

But that still does not explain why you would need a "cloud connected printer". All of it can be done locally.

You do not go on holiday and think "Oh hey, I need to put some stuff on my 3D printer". It is a solution that is desperately searching for a problem and creates a whole set of problems you do not want.

And for "I am not patient enough to wait for my print job to finish... Well, you could just start it in the morning and have it ready when you get home in the evening? That is, if you insist on leaving this stuff live and active without anyone in the immediate vicinity.

Neil Barnes

Well, you could just start it in the morning and have it ready when you get home in the evening?

Well, it works for sensibly slow bread (low yeast or sourdough types). Hmm, is there a marketing opportunity there? Cloud connected bread makers for extra fluffy bread? Oooh, 3-d printed extra fluffy cloudy bread? There's lots of scope for buzzword enthusiasm there!

Potemkine!

Maybe because the service provider can have a look to everything that is printed?

When the fog came in on little cat feet last night, it left these little
muddy paw prints on the hood of my car.