Why Are Silicon Shops So Shady Surrounding Specs?

[segaloco]

So I see it time and time again. Some device comes out, it has an address map where a bunch of components live, expose registers, buses, and so on. The goal of the manufacturer is to get people to buy and use their device. The manufacturer only makes that I/O map available to select developers under NDA. Any general person that gets their hands on that piece of hardware is then kneecapped from actually using it in isolation without combing over kernel source code and trial and error to figure out what registers mean.

What is the driving force behind this phenomenon? I feel like it's counterintuitive to getting people to buy your thing if you then restrict access (or don't provide access at all) to the critical information actually needed to make the device run. Sorry if this topic has been broached before but it's on my mind again with the chit chat in another thread about ARM system buses. Using the VideoCore VI as an example, Broadcom has not published how this works. I as an owner of a device powered a BCM2711 cannot fully utilize the product I purchased because I don't even truly *know* how it works. I just have to trust the tiny circle of folks with the NDA to see Broadcom's documentation to provide for me software that will work.

What is the impetus behind this nonsense? Is it software shops manipulating things to keep themselves relevant? Is it the fabs themselves thinking somehow telling nobody how to use a thing will attract them through mystique? Are they afraid their bottom line will crumble if a competitor knows they have a +5 Vcc pin somewhere? It's maddening, it creates a massive barrier to entry with, at least in my eyes, little demonstrable benefit. It's stuff like that that makes me wonder how the early, tight lipped Japanese video game industry took of at all. You had to jump through so many hoops to get access to information on how, say, the NES or Mega Drive worked. Doesn't the platform survive on people, ya know, producing content for it? Doesn't locking down that information artificially reduce the amount of people who can effectively use it? It's always been baffling to me. That's something I appreciate about AT&T's history, they documented so much and to such a degree that one could probably build a 1ESS from scratch given enough time. But God forbid I could even conceive of writing my own driver for any of the parts in a SBC I have.

 

[jlang]

Mostly it's to keep anybody from duplicating their part. They provide an abstraction layer (library)
to make the part function (under NDA)
They don't care if you can't use it. You won't buy a million of them anyway.

 

[daver2]

That is exactly the reason.

In order to duplicate a design, people will have to reverse engineer how it works. This is more difficult if there is no published documentation as to how it works.

Eventually, it will be reverse engineered. But, by that time, the financial impact on sales will be minimised.

At work we have descriptions of hardware, firmware and software under an NDA. We know how it works - because we write our own software to drive the hardware. But the terms of the NDA are such that we cannot reveal the contents.

The hardware is now obsolete though, so the technical details are now irrelevant. However, the NDA (and the legal terms and conditions still stand) until such time as the documentation is formally released by the owner into the public domain...

Dave

 

[Al Kossow]

the NDA (and the legal terms and conditions still stand) until such time as the documentation is formally released by the owner into the public domain...

Which never happens, because there is no business reason to do so.
Eventually the documentation retension date is reached, and it's all shredded.
Rinse, lather, repeat.

or... the data is sent to Rochester Electronics. But that is another story.
Eventually, they will be the only domestic silicon vendor left.

 

[daver2]

Correct!

Although I have been in a situation where one suppler has voluntarily transferred IPR to another supplier (that they were originally in competition with) because they wanted to drop that business line as sales had tailed off and they wanted to deploy their staff onto more profitable ventures.

Even though it wasn't put into the public domain, it was transferred to a competitor so the competitor could continue to support us on equipment they did not originally design or manufacture.

Dave

 

[cjs]

It's stuff like that that makes me wonder how the early, tight lipped Japanese video game industry took of at all. You had to jump through so many hoops to get access to information on how, say, the NES or Mega Drive worked. Doesn't the platform survive on people, ya know, producing content for it?

In this case at least, no. If you look at the reasons for the video game crash of '83, one of the major ones was having too many games; the quality got driven out by the crap. Nintendo succeeded not only because they marketed the NES more along the lines of a VCR than a video game system, but also because they set things up so it was difficult to publish a game without their approval, and they had quality standards for what they would approve.

We still have similar issues today. Look at any popular thing you might want to buy off Amazon, and probably you'll find it quite difficult to figure out which of the flood of options is at all half-way decent. And reviews don't help so much because they're gamed in the same way that the products are.

I reckon that the only real solution to this is some weird form of socialism where you can't get rich off your great ideas, and thus others who are focused on getting rich instead of great ideas have their motivation removed. We've seen it work in communities such as our own here. The problem is, being embedded in a capitalist society, we still all have to make a living.

 

[Bob-O-Rama]

There are a lot of reasons this stuff happens. Most of which would make no sense to normal reasonable people. Perhaps 20 years ago I would get contract jobs troubleshooting edge case performance issues with various vendor's NIC and storage hardware. They would basically comission me to figure out where the issue was, OS, driver, microcode, or hardware, and I would have dev tools for all these layers after they were mired down in 4-way finger pointing. You would get writers cramp from signing all the NDAs.

 

[segaloco]

Mostly it's to keep anybody from duplicating their part. They provide an abstraction layer (library)
to make the part function (under NDA)
They don't care if you can't use it. You won't buy a million of them anyway.

I'm not asking for a mask of the silicon so that I can analyze it on a transistor level, just what registers are there, what do you put in them to make things happen. They've decided oh, only these folks working on this specific driver for this specific OS kernel can be the arbiters of what is and isn't put on those registers, you filthy peasants don't get to effectively use your hardware without guardrails in place. It doesn't bode well for learning either, the less and less real, current examples you have to learn from, the more and more dated the general knowledge-base is going to become, with only select folks being in the "inner circle" that actually gets to know what the registers of some chip are. I hate the idea of flying blind and that I couldn't actually effectively audit most of the code sitting between me and my hardware. Do I need to all the time? Hopefully not. It at least makes me grateful that there are open source kernels where someone can at least try and reverse out as much of the hardware interface as that kernel uses. That being necessary at all though is still pretty stupid.

My two cents is if your only way to keep someone from stealing your supposedly inventive and bespoke design is to not let anyone know how to use it...maybe you need to R&D a little harder? I can't imagine something you can RE by just looking at the external interface has a whole lot of value add going on inside compared to other stuff on the market. What happened to setting yourself apart on quality rather than just being secretive. Maybe that was never a thing, but I've been lead to believe at least that computing used to be much more open, you could actually figure out how to program hardware peripherals directly. Now not so much.

 

[Chuck(G)]

Reminds me of the bad old days when PCs came with schematics.

 

[bakemono]

One issue is that competitors, who maybe already make a similar device, can then make a device compatible with yours at the register level. Now their part can replace yours in existing designs, or even run off your driver software (possibly excusing the competitor from needing to write a driver at all). There have been a number of freakouts over the latter situation, eg. "FTDIgate"

Another concern I've heard is that publishing less information, as a general rule, presents a smaller attack surface for patent owners to sniff out infringement.

Needless to say, it is unfortunate that we end up lacking documentation for hardware that we own, sometimes long after it is considered obsolete.

 

[Unknown_K]

Oddly enough you get schematics inside your microwave ovens.

 

[Hak Foo]

I suspect it's because the schematics/data books/reference material doesn't sell the product.

We as hobbyists might read the data sheets and decide which SoC works best for our needs, but when you're dealing with companies, it probably has a lot more to do with who your CEO golfs with that determines what gets added to your BOM.

Keeping the documentation OUT of broad circulation means that they can sell a turkey and by the time the developers finally look at it and determine it's an underdocumented, overpromise/underdeliver piece of rubbish, the deal's been struck, money has changed hands, promises made, and the sunk cost means you'll find a way to make that turkey fly at supersonic speeds with 125 passengers from London to Beijing.

I've seen this in my particular niche at work-- there are dozens of companies offering broadly interchangeable services in my vertical. The APIs range everywhere from "here's well-structured modern HTTPS/JSON with flexible handling of optional data and detailed response messages" to "here's a raw-socket format that actually uses Field Seperator, Group Seperator, and Unit Seperator characters intensively and unironically, requires you pass 25 empty fields if you need to populate field 26, offers no machine-readable status codes, only a semi-human-readable grunt limited to sixteen characters, and has undocumented failure modes where their server barfs out a complaint in EBCDIC." The customers buying the services don't know or care the difference, we just have to integrate it.

 

[segaloco]

It just seems to be a bandaid for a problem that could be solved in some more equitable way. Where is the accountability for the amount of waste and unusable stuff we have to deal with on a societal level due to it becoming both harder and harder to see the full picture of a system *and* systems being built with such short lifecycles in mind.

Maybe it would be justified if all the secrecy lead to more staying power in designs, but it seems to be the parts that are more public and standardized that stand the test of time. It just makes me worry too, so many resources are going towards just churning out new things all the time, sometimes with almost unnoticeable improvement, like new editions of a textbook...but so little in comparison are devoted towards then making any of it last. I get excited when I see open source hardware projects pop up, it would be nice if standard platforms could emerge from that sort of thing more often, like the Linux kernel but an equivalent in the realm of hardware. Plenty of folks make oodles of money off Linux, an organization could probably game a market strategy on open components without all the secrecy. Either way, I do hope this trend changes some day.

 

[Al Kossow]

With a few exceptions in the 21st century, companies absolutely do not care about products that are out of production.
In the case of Apple they actively change things so new software breaks on old hardware.
Combine that with the bloat and spying on the web, and you now have a perpetual hardware upgrade treadmill, where
all you see is enshitification.

One by one, every web site run by Microsoft has broken on older Firefox. I run "Little Snitch", an app that lets you monitor/filter
every IP connection going out of your machine, along with web filtering and the number of connections to Google and Microsoft
going on behind your back is disgusting. I have never touched anything coming out of Meta, I can only imagine what little lampreys
they are waiting to deploy.

 

[segaloco]

I'm glad to at least see I'm not the only one that finds it completely asinine. It's unfortunate though that I can't seem to break through to peers in my own generation that this current state of affairs is not how it has always been and not something that should just be accepted as normal. I have a pretty steady anti-smart-device drum beat going all the time, and it's funny because people will always admit oh yeah that sucks, I hate how social media and my cellphone and all of that are....and then they keep using them. That I think is one of the key problems, consumers just keep enabling this without demanding any better, and those of us who do are often shouted down with "Well that's just how it is!". How it is isn't how it needs to be, and complacency is certainly not the remedy. But if most people are complacent...sadly maybe it is what the world wants even if it isn't what I want. It just sucks being able to articulate why these things are all problems and still generally be dismissed by people who don't even grasp the full scope of the problem. That's not @ anyone here either, just folks I've discussed computing with out there in the world.