Licensing AI Engineers

The debate over professionalizing software engineers is decades old. (The basic idea is that, like lawyers and architects, there should be some professional licensing requirement for software engineers.) Here’s a law journal article recommending the same idea for AI engineers.

This Article proposes another way: professionalizing AI engineering. Require AI engineers to obtain licenses to build commercial AI products, push them to collaborate on scientifically-supported, domain-specific technical standards, and charge them with policing themselves. This Article’s proposal addresses AI harms at their inception, influencing the very engineering decisions that give rise to them in the first place. By wresting control over information and system design away from companies and handing it to AI engineers, professionalization engenders trustworthy AI by design. Beyond recommending the specific policy solution of professionalization, this Article seeks to shift the discourse on AI away from an emphasis on light-touch, ex post solutions that address already-created products to a greater focus on ex ante controls that precede AI development. We’ve used this playbook before in fields requiring a high level of expertise where a duty to the public welfare must trump business motivations. What if, like doctors, AI engineers also vowed to do no harm?

I have mixed feelings about the idea. I can see the appeal, but it never seemed feasible. I’m not sure it’s feasible today.

Tags: , , ,

Posted on March 25, 2024 at 7:04 AM • 43 Comments

Comments

Stephen • March 25, 2024 8:02 AM

Physician and attorney self governance are both organized at the state level. There are advantages to operation at this scale – chiefly that smaller communities will tend to know their members better. Another is that these professionals are relatively evenly distributed, relative to population and very visible within society. I don’t think that AI self governance is workable within that model. Designers are geographically concentrated and can perform their roles with little or no contact with the public. There are incentives to perform work in secret that do not influence the practice of law or medicine and there are few visible reputational harms that arise from malpractice. A national licensing and self-governance system would therefore rely on the best intentions of members who are poorly incentivized to act ethically.

Petri Aukia • March 25, 2024 8:09 AM

Licensing is an approach that works well, once the field is relatively slowly moving and we have consensus on the best practices in the field. We may be in that position for many subfields of software development, but in any rapidly moving field this would entail teaching and reviewing skills that are no-longer relevant to the mainstream in the industry.

Clive Robinson • March 25, 2024 8:34 AM

@ Bruce, ALL,

Re : Pull up the draw bridge.

“I have mixed feelings about the idea. I can see the appeal, but it never seemed feasible. I’m not sure it’s feasible today.

It will never be feasible in any domain because it automatically creates a “closed shop” that can then be exploited by insiders against outsiders.

All through history we have prime examples of how “closed shops” are used to “limit supply” below demand. Thus forcing a “Cartel or Monopoly” that then gets “exploited”

Such monopolies create,

1, High costs for customers
2, Indentured service for practitioners
3, Lazy non development cultures.

We know this not just from ancient history of “The Guilds” but all the “pull up the draw bridge” nonsense we see in Trade (think Uber et al) and Telecoms already. And why the likes of AT&T were “broken up” but unfortunately insufficiently to stop them becoming a cartel in replacement of a monopoly.

Trust me when I say the cost of such “Regulated Professions” significantly out weighs any of the perceived advantages by a very very long way.

Passerby • March 25, 2024 8:41 AM

How would you even enforce that? I can download a model from anywhere in the world, made by anyone in the world, and run it. I can also build one however I want, and upload it or share it p2p. How are you going to exert any control whatsoever on the engineers building the models?

You may be able to restrict public Government use of these models, but that’s not where bad uses will happen, not for long anyway. Individuals and private institutions working in secret, and also bad actors and secretive public institutions, that’s where things can go wrong undetected, indefinitely. Guild-era solutions may work when the work is done in a public/semi-public setting where the “artisan” is exposed while doing the job.

Daniel • March 25, 2024 8:50 AM

It works well in the pharma and medical devices industries: I suppose that when lives are in scope, in any field of engineering, regulatory supervision can only help. Extend that to food, environment and so on, and the balance can be tipped the right way(pun intended).

Clive Robinson • March 25, 2024 9:06 AM

@ Daniel, ALL,

It works well in the pharma and medical devices industries

No it does not. I’ll let others list the oh so many failings in that niche area over just the last five years.

But as it’s in the MSM currently, ask yourself why after strong –but supposedly– independent Federal regulatory control in aircraft manufacture was at the demands of the industry became “self regulated” did Boeing start getting into the “manslaughter by negligence game”. With the US DoJ and FBI finally acting in investigating Boeing and it’s management.

And before anyone claims Boeing is a “rouge example” that is very far from the case, every fractional part of every industry that exists has “rouge operators”.

It’s due to the mentality of certain types of people that represent something like 10% of the actual population of which around 10% of those are actual convicted criminals…

John Beattie • March 25, 2024 9:25 AM

It can be enforced: use engineer status to enforce insurance requirements.

The issue, for people in the English-speaking world, is this: Continental Europe regards engineers as professionals in the same way as doctors and lawyers. The UK, US, Canada don’t. That would have to change.

tim • March 25, 2024 10:00 AM

@john

Define “engineer”. While we are waiting for that – exactly what are we pulling insurance for?

Belkin • March 25, 2024 10:53 AM

… “Licensing” ultimately means obtaining government-permission to work at some profession– it has both benefits and big negative consequences generally

private professional “Certification” of lob qualifications is a much better option that eliminates the major drawbacks inherent in arbitrary government control of any profession

JonKnowsNothing • March 25, 2024 12:02 PM

@Daniel, @Clive ALL,

re: Medical and other Licensing

Licensing creates an artificial shortage, it is designed to restrict a resource and therefore increase scarcity and salary or profit. It doesn’t matter what type of restriction whether it’s a University Degree, Certificate or Exam license system, it’s all an attempt to create a faux-shortage.

In South Korea, an attempt to increase the number of medical student slots at Medical Schools there has generated a strike. The government wants to add 2,000 extra students per year to their medical school system. The professors, MDs and Trainee MDs all know this will depress their earnings. (1)

In the UK, there is also a rolling strike by Trainee MDs and Senior MDs in support, because they are paid a tiny wage for years and years before they “qualify”. The block is getting Qualified and the UK Gov uses this to suppress the Trainee wages because they have no other path available.

It’s the same with all such qualifications.

An old joke:

  • What do you call the lowest rated-ranked medical student of their class at graduation?
    • Doctor

===

1)

htt ps://w ww.thegu ardian.com/world/2024/mar/25/south-korea-doctors-strike-protests

  • South Korea doctors’ strike widens as medical professors join protests

  • “It is clear that increasing medical school admissions will not only ruin medical school education but cause our country’s healthcare system to collapse.”

  • a plan to increase the number of students admitted each year to medical school from 2025 to address shortages in rural areas and greater demand on services caused by South Korea’s rapidly ageing population.

The USA has-had a special program for medical students to waive their medical school costs if they worked in disadvantaged areas.

It doesn’t work well because people who live in Big Urban Centers and can muster the scores to get into medical school do not want to live in a Dry Camp location.

A funny movie that includes this situation in the plot:

ht tps://en .wik ipedia.org/wiki/The_Unknown_Saint

  • The Unknown Saint is a 2019 internationally co-produced comedy-drama film directed by Alaa Eddine Aljem

Morley • March 25, 2024 12:05 PM

Lots of talk about what needs to be done, not as much on how to make society do it. Is there an area of study on wide social engineering specific to our current world? Closest I can think is education and propaganda, but that’s not specific.

cybershow • March 25, 2024 12:30 PM

While I generally approve of regulation of digital technologies with massive social
impacts, with this I see problems around foundations, granularity and representation.

It’s like we’re getting ahead of ourselves and assuming the basics are settled. They
are not.

Despite ambitious and thoughtful work on the EU AI Act I don’t think we have even the
most elementary moral foundations worked out for this yet. Not only have we not found
all the answers, we have not even formulated the right questions. This does not seem a
good basis from which to go about handing out licenses.

Doctors and lawyers come from a much older, deeper ethical lineage. Software
Engineering is about 50 years old. We haven’t worked out whether it’s okay to throw the
mental health of a generation of children under the bus for the “convenience” of
smartphone tracking and financial legibility. How are we now to ponder something
immeasurably more morally complex and far reaching?

With respect to granularity, there’s no uniform definition for AI. As a DSP engineer,
would I run into problems implementing a filter for a motor control because it had
three instead of two Markov orders? Because some academic once misused the word AI in
a paper title to get in the trendy journal? Would a colleague be allowed to work
unlicensed on a procedural solution to the same problem?

And who is this really for? Other than by tortuous hierarchies of representation where
is the will of the people in this?

Being pro government/social contract in my thinking around steering technology doesn’t
blind me to the fact that we’re in an age where all governments seem increasingly
disconnected from the will of the people and are in dubious extra-national alliances
with big-tech monopolies. To the point of shameless corruption and treachery in many
cases. I care little whether my human rights are being abused by government approved
machines or ones built by commercial monopolies. It doesn’t hurt me any less if I’m at
the sharp end of weapons built by licensed engineers.

Licensing people to produce highly complex, intrinsically questionable things which
have deferred functionality does not make them any less problematic. It does not
address the questions. It is stamping assent, not cultivating debate and choice around
the real ethical issues in technology today? At worst it is handing legitimacy as
a substitute for more difficult decisions.

Foremost, who is it for? And why? To what end?

That leads to – what say do we get in it?

I don’t see a whole lotta love
or much reflection in this
area. Just a lot of vying over future power.

fib • March 25, 2024 1:21 PM

The underlying argument is that at any time an AGI can ’emerge’ from neural networks, with the additional argument that neural networks are “mysterious” [a really lame word to use in the context of science] and that “no one knows what goes on inside”. This is false on several levels. Neural networks are still computed in a classical way, like banking systems. The difference is that in the intermediate phase of the process huge matrices are manipulated, and, in fact, it is not humanly possible to know the exact content of these matrices, which is no surprise. The tensors of neural networks during processing are not the only example of unknowable entities. There are countless unknowables in quantum mechanics [infinities], for example, which does not prevent us from taking practical advantage of the area. Nobody fears the emergence of a vacuum decay when two electrons interact.

Adrian Gropper • March 25, 2024 1:27 PM

This is a very good paper and the comments on feasibility make sense. However, both the paper and the feasibility comments presume professionalization at the product development stage. The Hippocratic Oath was not meant for corporate employees.

Let’s not ignore the potential for professionalizing engineers as fiduciaries in the use of large language models and other AI. This would be analogous to the way individual physicians are credentialed to apply the product of corporate entities like pharma and hospitals to individual patients as clients.

Clive Robinson • March 25, 2024 1:58 PM

John Beattie, ALL,

Re : Worse than payed gig-work.

“It can be enforced: use engineer status to enforce insurance requirements.”

Sorry that is a recipe for disaster as I noted above with,

“Such monopolies create,

1, High costs for customers
2, Indentured service for practitioners
3, Lazy non development cultures.
“

You see that second point?

Look into the accountancy profession as to,

A, Who does all the work.
B, Who takes home all the fees.

You will find “the workers” get at best peanuts, whilst “the signers” get the “Big cut of the Partnership fees”.

We’ve seen the IEE and BCS try to get this “Chartered Engineer” nonsense for years. However most in the industry have realised they will be the ones not getting the elevator car to the top floor, but the shaft down to the Stygian pit at the bottom way way below the comparative delights of Hell.

Agammamon • March 25, 2024 1:59 PM

If the potential dangers of AI are existential – human extinction – then how would requiring a license help if there are other nations that do not?

It seems like this would be a good way to ensure we don’t cause the extinction of humanity – while not actually reducing the risk that someone will.

Anything that isn’t effectively a global regime – with the ability to enforce its rulings by military force – is ineffective. Think of the attempts to force the West to reduce carbon emissions while ignoring China and India’s contributions, let alone the rest of the developing world as it develops and, in the end, being completely ineffective at stopping climate change because of that.

Agammamon • March 25, 2024 2:04 PM

Look at some real world examples.

Human cloning was made really hard to do legally in the US during the 1st Bush Admin – so cloning research moved to South Korea (until their big scandal got them to change their rules).

Recycling and disposal regulations not only increased fly-tipping in the UK but in most of the West got us to start shipping garbage across the ocean until China said they wouldn’t take it anymore.

Or the big one.

The US made ‘gain-of-function’ research effectively illegal in the US – so the US moved that research to China. And we know how that turned out.

Winter • March 25, 2024 2:18 PM

@JonKnowsNothing

Licensing creates an artificial shortage, it is designed to restrict a resource and therefore increase scarcity and salary or profit

A driver’s license ensures that those driving a car have at least a minimum level of expertise. That minimum level differs per country, but is relevant. I don’t think it is exclusive. Those who want to get a driver’s license can do so. Those who fail are unlikely to be fit to drive a car safely on public roads

A doctor’s licence ensures the owner has a minimum level of expertise in human health. It is not exclusice as very few people without this license would be fit to do a doctor’s job. I doubt you would want to be treated by someone without a MD degree. It is exclusive in that getting access to the training is very expensive, prohibitively so. But that is a problem of US education.

Clive Robinson • March 25, 2024 2:24 PM

@ cybershow,

Re : We are nowhere close.

“I don’t think we have even the most elementary moral foundations worked out for this yet. Not only have we not found all the answers, we have not even formulated the right questions. This does not seem a good basis from which to go about handing out licenses.”

We are not even remotely close to asking questions.

As you know I’ve a certain view point on this current AI nonsense. Which to be polite is, that is engineered to see a parting of the ways between individuals with a lot more money than sense and their money.

Much though I would enjoy seeing such types taken down a few trillion, the simple fact is it would under normal circumstances be considered fraud as it’s just another “pump-n-dump” by the “Venture Capitalists”(VC’s).

More insidious however is what Google and Microsoft are upto.

As you are aware I regard the current AI systems as being the most powerful surveillance tools so far developed.

What Microsoft and Google are doing is “running interference” to buy time. They know if they can get there systems into place before any Government can act, they will in effect “Get a free pass” whilst others will have to jump through a the hoops etc.

It’s a trick Uber tried and got away with mostly. It’s only now some half decade or more later that Governments are seeing the harms not just to the workers but society in general and so are now trying to rectify them against significant entrenched and well funded “push back” from Uber.

You can make a fair bet that this “play” is in Microsoft and Google’s unpublished business plans.

Steve • March 25, 2024 3:02 PM

@Clive Robinson:

It will never be feasible in any domain because it automatically creates a “closed shop” that can then be exploited by insiders against outsiders.

Yeah, anyone should be allowed to practice medicine or civil engineering.

Why bother with years of schooling and residency when you can just hang up a shingle and start prescribing and doing surgery?

And who cares if the person who designs a highway bridge has had any sort of education or knows a trestle from a tea kettle?

Let the free market decide who the better practitioner is.

While, sadly, licensing doesn’t keep all the incompetents out of a field, I suspect it does tend to winnow the majority of them out.

Sigh

Aaron • March 25, 2024 3:24 PM

NO to licensing!

What have we learned about licensing people under the guise of blessing them as acceptable individuals within their licensed community?
1) It’s a subscription model of wealth generation for those who provide licensing services and management
2) The policies that start with a licensing agency are rarely the ones that remain the longest within the licensing parameters, scope creep!
3) It’s a secondary model for controlling the industry you are licensing. Can’t find something legally wrong with a person to fire them for, then you can certainly make up a reason to yank their license to work.
4) Licensing is socially treated as expertise, which is a big gamble. Good actors are unlicensed and bad actors are licensed; only time can distinguish the two.
5) Gatekeeping and tier based acceptance become inherent in the system. Don’t like the “type” of people being licensed, add one more caveat to make it more difficult to achieve.
6) If, by the same metric certain politicians keep and spout around election season, licensing costs money, it is therefor “racist”

We license alcohol distribution and yet minors still acquire it.
We license drivers and yet the worst drivers still have theirs.
We license aviators and still have intoxicated pilots.
We license health care professional and look at the state of health care in America.
We license the real estate industry and look at what they enabled leading up to 2008?

Licensing does not fix problems, not can it prevent them.
Companies need to be harsh and draw a hard line; if you can’t perform to a certain standard, say good bye. Stop coddling employees for the sake of preventing a fictionalized lawsuit or move your company HQ to a state that is right to work to avoid most wrongful termination lawsuits. The past decade of HR pushing for more DEI is the opposite stance to ensure good quality employees. Good distinguishes itself, you rarely get “good” when you force it, especially by creating identity quotas.

JonKnowsNothing • March 25, 2024 3:29 PM

@Winter, All

re: very few people without this license would be fit to do a doctor’s job. I doubt you would want to be treated by someone without a MD degree. It is exclusive in that getting access to the training is very expensive, prohibitively so.

Nearly everyone with an internet connection does a Search for their medical condition or whatever condition they think they have.

Every Health Care organization in the USA, hands out a Self Care manual (hard copy or pdf), with links to website listings of conditions, treatments, explanations. It includes details on how to read your own labs, xrays and imaging reports. There is a section called : What does this mean or Learn More.

Nearly every health system in the western economic sphere has to import health care workers. In the UK, ~9,000 imported nurses are leaving the UK after ~3yrs there to go Anywhere Else except stay in the UK. The USA, is very happy to see them.

Inside US clinics, there is a whole phalanx of people doing “medical stuff”, they take your BP, HR, Weight, go over the check list of your meds and conditions along with “What are you here for today”.

I see a lot of people on every clinic visit that are not MDs and not trained as RNs but they know how to tick a box on a form.

If you call the Health Organization Urgent Phone number, the first person you talk to is not a MD. They follow a prompt script. If you get to the bottom of the script you might get transferred to the RN of the Day, who may if your condition warrants, talk to the MD on Call (but you don’t get to talk to them) and then they will refer you to

  • The self-care section on [topic] (free)

or

  • Goto the ER ($$$)

re: Drivers Licenses

This also is a gating feature. It was not a requirement for a very long time. It did help the government get a list of names, ages, locations of the male population especially for Military purposes.

It does not in any way guarantee that the person with the license knows anything. Lots of people drive without them (illegally) some because they cannot afford the proof of insurance forms, the license fee costs, and some because they never got one. There are a lot of reasons.

Currently, it is used to restrict access to buildings, getting an apartment, having a bank account, getting a job, buying a plane ticket (RealID). All of these have nothing to do with driving a vehicle.

iirc(badly) Back before the USA had diplomatic relationships with China, and before the expansion of their economy, reports were that people had to be able to repair their own cars. They had to show how to take apart the carburetor, and perform various mechanical repairs as part of the drivers licensing program.

Farmers still know how to do this because they often have old equipment that Does The Job. Ancient memory of Venturi valves.

===

ht tps:/ /en.wik ipedia.org/wiki/Carburetor

ht tps://en.w ikipedia.org/wiki/Venturi_effect

  • The Venturi effect is the reduction in fluid pressure that results when a moving fluid speeds up as it flows through a constricted section (or choke) of a pipe.

Escaped the Moderator • March 25, 2024 5:17 PM

You don’t need licensing, you need regulation.

Adam Smith pointed out that markets require regulation.

…markets and trade are, in principle, good things—provided there is competition and a regulatory framework that prevents ruthless selfishness, greed and rapacity from leading to socially harmful outcomes

You shouldn’t need a taxi-driver’s medallion to be a taxi-driver: you should need to show a regulator that you meet the necessary requirements – the medallion simply skews the market. Licences are medallions, sometimes obtained by non-pecuniary means.

What is needed are regulators that cannot be captured, and a properly enforced and publicly audited regulatory framework – but finding incorruptible expert regulators is hard.

vas pup • March 25, 2024 5:49 PM

For government work – feasible, for private companies – not.
As you, Bruce, suggested in the past – solution is liability insurance, all other will follow.

Anonymous • March 25, 2024 7:53 PM

@Escaped the Moderator – yes, something along what you said: regulation is what would be required, with all the implied good and bad apples. Can’t see any other (feasible) way, really.

lurker • March 25, 2024 9:58 PM

@tim
“– exactly what are we pulling insurance for?”

So the insurance brokers can retire in comfort. They don’t care if the engineers are licensed or not so long as they keep clipping the ticket.

Arclight • March 26, 2024 2:48 AM

The other issue here is that large software projects and related research typically require the efforts of many people, have inputs that are in the case of AI, are almost impossible to catalog and processes that are sort of opaque. There is no easy way to have an individual “review and stamp” the plans or meaningfully inspect the final product the way a bridge engineer might be able to.

Clive Robinson • March 26, 2024 7:08 AM

@ Arclight

Re : To see the light eyes have to be open

“The other issue here is that large software projects and related research typically require the efforts of many people, have inputs that are in the case of AI, are almost impossible to catalog and processes that are sort of opaque.”

Also they become “endless jobs” where work continues on or with them endlessly.

These things we would hope are obvious…

But it appears that the consequences of this,

“There is no easy way to have an individual “review and stamp” the plans or meaningfully inspect the final product the way a bridge engineer might be able to.”

As you and I realise, appears not to have got through to as many as it should have done.

Worryingly though Politicians appear not to have grasped the notion of “open ended” science and engineering yet. Heaven alone knows what mess they are going to create…

I suspect that they “will take advise” from the “usual suspects” of Silicon Valley Mega Corps…

Thus we will hear the creak of draw bridges being lifted.

ImGettingTooOldForThis • March 26, 2024 4:15 PM

Oy vey!

Look at software engineers. Do they have veto power over bad designs? Nope. They gotta do what they’re told.

Look at Chief Security Officers (CSOs). Do they get the budget and power to protect the company? Noooo. Do they have the power to report hacking break-ins as required by law? Noooo. Do they get arrested and face jail time if their company is hacked and their superiors don’t let them report? Yessss.

Licensing AI professionals works the same way. Offload the risk & blame onto some poor sap so the owners and other C-level Execs are protected. It’s a lot cheaper than doing things right.

cls • March 27, 2024 1:59 AM

surprised no one posted this yet:

Software engineers will never be Professional Engineers as conventionally understood in the US, because of the halting problem.

All non-trivial software has defects. it’s a fundamental law of the universe. Can never get all the bugs out. Formal methods don’t help.

I wonder what ACM, and IEEE, have to say about this?

Clive Robinson • March 27, 2024 4:29 AM

@ CLS,

Re : Halting Problem

“Software engineers will never be Professional Engineers as conventionally understood in the US, because of the halting problem.”

The “Halting Problem” is not a “software” issue, but an issue to do with systems complexity.

Therefore it will appear in any system that meets certain basic logical requirements not just “software”.

“All non-trivial software has defects. it’s a fundamental law of the universe. Can never get all the bugs out.”

Actually not true. The underlying system is deterministic and has a finite number of states it can be in. Provided you correctly manage state transitions it’s behaviour will be known and predictable.

The “Turing Engine” is a finite state machine and fully predictable in it’s actions.

What is not fully predictable in time less than transitioning through all the input states is the machine transitioning into any given state.

Because of the “infinite tape” input and “feedback” allowing modification of the tape thus the state machine behaviour beyond that which is predictable in any less than transitioning through a sequence of all the potential input states.

There is not the space here to go through it all but imagine a system that implements an accumulating counter that can roll over. It will go on forever as there is no stop state given. So we add a simple compare on value so if the accumulated count matches the value it halts. This is a simple fully deterministic system. Now consider if it will reach the stop value or not. If we start at an even value and add an even value then below the rollover it will only contain even values thus if the match value is odd we implicitly know it will not stop before rollover. We further know if the rollover does not have the effect of adding an odd value then we further implicitly know it will not stop after rollover.

Now consider that the rollover has the effect of adding half the value held in the accumulator prior to the rollover it’s still a fully deterministic system but determining if it will stop or not becomes a whole lot harder.

We believe from other work on pseudo random sequence generators there is no easy way to make a prediction if any given stop value will ever appear, with any less effort than of going through every state, even though it is a fully deterministic system.

Jacob Brodsky • March 27, 2024 8:05 AM

Historically, we have required licenses where the practice of a profession could place the public’s lives in danger. Thus we license drivers. We license marine and aviation pilots. We license Medical Doctors, Engineers, and… Attorneys? (whatever –I sometimes have a bad opinion of those who presume exclusive knowledge of the laws we are all supposed to understand and abide by)

What licensing does is give a state a legal instrument to prevent and prosecute unqualified or reckless people from doing certain activities that may jeopardize public health or wellbeing. For example, when designing a bridge, there are many subtle things that could go wrong. We do our best to ensure that the Engineer is licensed, aware of the issues, and held liable for the design.

Thus, my question: What would AI software be applied to that might jeopardize the public’s wellbeing? Why wouldn’t existing licensing laws be sufficient? Answer those questions and the conversation could continue.

Clive Robinson • March 27, 2024 9:43 AM

@ Jacob Brodsky, ALL,

Re : Approvals is a full disclosure process.

“when designing a bridge, there are many subtle things that could go wrong. We do our best to ensure that the Engineer is licensed, aware of the issues, and held liable for the design.”

The process is actually not as you describe it.

I’ve never had my status as an Engineer questioned, nor have I had people quizz me about the legalities or engineering details I guess in part because they themselves are insufficiently qualified to do so. Even when working abroad where they have quaint pull up the draw-bridge legislation where such rules as must be state registered for at least five years before you can call yourself an “engineer” are just ignored. Usually they simply use a different job title such as Project Director rather than Senior Engineer. Stick the word Manager or Director in and the legislation and regulation falls into abeyance.

Something that happens all over the US and as far as I can tell Europe as well. As for “liability” I’m guessing you’ve not played the “Subcontractor Game”. Liability is corporate not personal and is carried by insurance.

The important thing to remember though is “Open Disclosure” the designs for a bridge or any other item subject to legislation or regulation is that it has to go through an independent approvals process and in actual “engineering terms” that process has to be “Open to public scrutiny”…

At the moment because “software” is seen as a part of a system that is a sub-component it can be in effect hidden away from the certification process in various ways. Although that is changing all be it slowly since the Log4J debacle back in 2021,

https://theconversation.com/what-is-log4j-a-cybersecurity-expert-explains-the-latest-internet-vulnerability-how-bad-it-is-and-whats-at-stake-173896

(Even Whitehouse Executive directives are little match for various management types in Silicon Valley Mega Corps and the like).

The problem is for some “systems” the reality is,

1, The software is the System.
2, There is no recognisable engineering design process.
3, There is nothing that can be engineering standard audited.
4, The system is never completed.
5, The software is considered a trade secret.
6, The software is the Corps only “competitive advantage”.

There is no way they are going to open up to an external process let alone one with a publicly “Open Disclosure” process in place, without a very protracted battle.

But the problem with AI systems is “the training data” as the old saying has it,

“Neither fish nor fowl”

So how do you licence it, and it’s practitioners? And will their management allow it to happen?

Have a look at what Elon Musk is trying to do over Open-AI and the Microsoft server farm…

Sasha • March 27, 2024 6:32 PM

@ ImGettingTooOldForThis,

Look at software engineers. Do they have veto power over bad designs? Nope.

Then they’re not software “engineers”. I’m aware that programmers in the U.S.A. and elsewhere have been getting away with using that term, while non-rigorously hacking stuff together and otherwise ignoring normal engineering practice. “Real” engineering documents, though, are marked with a professional seal, and an engineer’s employer cannot compel sealing.

A lot of software doesn’t really need that level of rigor. It’s not clear to me why “A.I.” should be the dividing line; and with SSRN demanding that I “enable cookies and Javascript” I guess I’m not going to bother downloading the paper. Certain uses of A.I., sure; if it’s controlling a machine that could kill someone, like a Tesla car, of course the engineering should be real. If it’s drawing pictures based on a prompt and a database of published data? Probably not. For something like a web browser, maybe the sandboxing code and any code that runs outside it should be the boundary; whatever shit that runs inside a properly-engineered sandbox can continue to be shit.

JonKnowsNothing • March 27, 2024 7:15 PM

@Sasha, All

re: Real Engineers v Software Engineers

This argument was lost a long time ago. You are beating a dead horse.

Don’t feel bad, USA Grammar Folks are dismayed because they lost the argument years and years ago over Lay and Lie.

The Oxford Comma though is still in play for complaints.

(1)

I’m gonna lay down my sword and shield,

(A-ha) down by the riverside

I ain’t a gonna study war no more

(2)

We’ll fight for our country, our King and his crown,

And make all the traitors and croppies lie down.

Down, down, croppies lie down.

===

1)

ht tps://e n.wikip edia.org/wiki/Down_By_The_Riverside

  • “Down by the Riverside” (also known as “Ain’t Gonna Study War No More” and “Gonna lay down my burden”) is an African-American spiritual. Its roots date back to before the American Civil War, though it was first published in 1918

2)

ht tps://en.wikip edia.org/wiki/Croppies_Lie_Down

  • “Croppies Lie Down” is an Irish folk song, dating from the 1798 rebellion in Ireland, that celebrates the defeat of the Irish rebels.
  • “Croppies” meant people with closely cropped hair, a fashion associated with the French revolutionaries, in contrast to the wigs favoured by the aristocracy.

Jacob Brodsky • March 27, 2024 8:50 PM

@Clive Robinson: I am a practicing professional engineer, and I have been licensed for over 20 years. I say practicing, because nobody is perfect. I’ve been an engineer for over 30 years.

When I sign and stamp a drawing, I assume personal liability for the safety of that design. Note my words carefully. How it gets built and the adjustments made during that time are a different story. As long as the execution of that design matches the specifications and drawings, the builder will not be liable. If the general contractor or sub-contractor does not build the critical sections of the design as drawn or specified, they will be held liable.

Most, if not all states require certain documents, such as the design for a large boiler, to be certified by a Professional Engineer. The reasoning is that if that boiler explodes, it will probably kill people. We need to make damned sure that first, the engineer is sufficiently educated and experienced through certification; and second that he or she willingly assumes personal legal liability for the design. That keeps people with other titles at bay.

But the failure of most software usually does not immediately risk life or limb. I say “most” because sometimes there may be software in a life safety system. As the laws stand today, Engineers take on the liability for that software. Most of them have no capacity to evaluate the software, so there are specifications for testing and proof of performance such as IEC 61508 and IEC 61511 (Similarly ISA has ISA-84).

The NCEES did have a category for Software Engineer of critical embedded systems. But so little interest was shown for that certificate that they stopped examining anyone for it several years ago.

But as for licensing the development of AI for a life safety system, I will not sign off on anything like that, nor will most other professional engineers. And I warn anyone contemplating the use of AI for life safety applications to carefully consider whether the use is warranted and necessary.

Thus I see no point in licensing anything having to do with office software or AI in particular.

JonKnowsNothing • March 27, 2024 10:21 PM

@Jacob Brodsky, All

re:When I sign and stamp a drawing, I assume personal liability for the safety of that design. Note my words carefully. How it gets built and the adjustments made during that time are a different story. As long as the execution of that design matches the specifications and drawings, the builder will not be liable.

I’ve known a fair few Software Designers who use the same sad excuses when their design fails.

  • Big Dog: Did you follow all the instructions?
  • Little Dog: Well not this section, ’cause it didn’t make sense.
    • End discussion.

The few I’ve known who use this method, palm off their faulty design which is why it doesn’t make sense. However, they have enough mojo to convince ZBoss that it will work.

Then the rest of the Little Dogs get blamed cause it doesn’t work and we have to come up with an acceptable work around because the project was relying on a faulty design and there’s never enough time to do it over properly. Mostly because there isn’t enough $$$ left in the project and we have to deliver something.

In the real world there’s things like

  • Grenfell Tower (1)

And in software there is

  • Meta/Facebook MITM attack to decrypt the encrypted messages of rival companies (2)

Do not pretend your attitude absolves you of anything or that your attitude condemns others because they do not have your superior ability to crush a rubber stamp on a piece of paper.

===

1)

ht tps://e n.wik ipedia.org/wiki/Grenfell_tower

ht tps: //e n.w ikipedia.org/wiki/Grenfell_Tower_fire

  • On 14 June 2017, a high-rise fire broke out in the 24-storey Grenfell Tower block of flats in North Kensington, West London, at 00:54 BST and burned for 60 hours. Seventy people died at the scene, and two people died later in hospital, with more than 70 injured and 223 escaping.

  • The fire was started by an electrical fault in a refrigerator on the fourth floor. This spread rapidly up the building’s exterior, bringing flames and smoke to all residential floors, accelerated by dangerously combustible aluminium composite cladding and external insulation, with an air gap between them enabling the stack effect.

2)

HAIL Warning

ht tps:/ /arst echnica.com/tech-policy/2024/03/facebook-secretly-spied-on-snapchat-usage-to-confuse-advertisers-court-docs-say/

  • Facebook secretly spied on Snapchat usage to confuse advertisers, court docs say
    Zuckerberg told execs to “figure out” how to spy on encrypted Snapchat traffic.
  • a secret Facebook project initially called “Ghostbusters,” designed to sneakily access encrypted Snapchat usage data
  • It worked by “develop[ing] ‘kits’ that can be installed on iOS and Android that intercept traffic for specific sub-domains, allowing us to read what would otherwise be encrypted traffic so we can measure in-app usage”

Jacob Brodsky • March 28, 2024 5:02 PM

@JonKnowsNothing Before I was an engineer I was a technician. So I know of what you speak. Yes, certified people are still human and they do make mistakes, just as Doctors, Pilots, Master Electricians, and every other professional service. What you’re supposed to do when you don’t understand something is ask questions. If you don’t understand the answers ask more questions. If you can’t get a reasonable answer, stop work and wait until they provide an actionable answer.

Certification does not prevent such disasters. What it does is mandate minimal education and training qualification so that certificate holders have an opportunity to understand the issues, an apprenticeship to learn the practicalities and ethics of the business, and also a legal handle of liability for one’s actions. That’s as good as any certificate gets. It’s an additional tool for ensuring that the person in charge of a life-safety issue has every reasonable chance of being competent.

Is it perfect? No. One of my relatives is an attorney who has prosecuted professional engineers. He has some awful stories to tell. Some people just don’t care and that attitude pervades everything in their personal and professional lives. But the key take-away is that they were prosecuted for professional misconduct and had to pay significant fines, in addition to having their license to practice in the state revoked.

Nevertheless, the point of the certification is life safety. Right now, AI is not connected to any significant life safety systems. I think will be quite a while before anyone allows such systems to be used for life-safety purposes. As such, I would agree that certification is not helpful for most software or AI.

As for your attitude about Professional Engineers, I believe you could say the same for most other licensed professionals. Nobody is perfect. But we try to ensure that they have a reasonable education and experience before we allow someone to assert authority. That’s all we can do.

Anonymous • March 28, 2024 6:59 PM

Software engineers don’t need licensing. Competence and excellence are obvious. This is just a way to destroy the freedom enjoy by those who learn to code. You gonna stop me from writing and distributing SW without a GOV permit? GFY.

JonKnowsNothing • March 28, 2024 7:43 PM

@Jacob Brodsky, All

re: If you don’t understand the answers ask more questions. If you can’t get a reasonable answer, stop work and wait until they provide an actionable answer.

It is clear you never worked for a large corporation. Maybe not even a medium size corporation. Or, perhaps you got chums in the board room who keep you on the payroll even though you are not doing your job.

re: Right now, AI is not connected to any significant life safety systems

You have not been keeping up with the literature or the implementation. There are several types of life safety systems already affected

  • AI Software driven selection points in the civil area. This is anything from banking, to the price of bread in your market. This includes using AI to assist in civil engineering design development. It is not limited to the standard building modeling for wind, earthquake, flood, fire, heat, cold resiliency but includes material selection based on the AI input query.
  • AI Software driven selection points in the medical and pharmaceutical industry. This is anything between Hospital Admitting, lab work, imaging, even down the the hospital feeding schedule in the kitchens. When a fiancial threshold is crossed it is called Triage and your choices fall to zero.

AI is all around you but perhaps you haven’t noticed it yet. A “smart meter” ROBODEBT demand might clear the fog some.

A Nonny Bunny • March 30, 2024 4:28 PM

<

blockquote>What if, like doctors, AI engineers also vowed to do no harm?

<

blockquote>

Well, it’s always a trade-off isn’t it. A doctor may need to cut you to heal you. And an AI engineer might need to create a super-intelligence to enslave guide humanity to save it from itself.
People never have much trouble justifying things for the greater good.

JonKnowsNothing • March 30, 2024 5:46 PM

@All

re: vowed to do no harm

USA The “Hippocratic Oath” is a statement of ethics, not of law. It’s states an ideal state not necessarily the practical state.

It’s why the top torturers in the CIA black sites are medical personnel. The most common person administering the torture are women MDs and professionals.

  • Women are preferred due to the cultural backgrounds of the people they torture

If you transgress into “medical malpractice”, going against the “usual and customary procedures”, you can end up in a court of law.

  • The CIA protects their medical torturers from any legal consequences

Having a medical license rescinded is similar to a lawyer being disbarred; it is done by the certifying organizations.

  • In one of the dust ups resulting from the discovery that US Licensed Medical persons where involved in the design and implementation of the 911-Torture Manuals both the theory and practice, the only consequence was a long fight within the certifying organization over their license. For many years their license was upheld and maintained in “good standing”. Years later their licenses were revoked or rescinded. However both had retired by then, so it was not a problem. They were able to maintain their medical practices up to that point, treating people who had “No Idea”.

===

  • ht tps:/ /en.wikipedia.org/wiki/Hypocratic_oath
  • The Hippocratic Oath is an oath of ethics
  • There is no direct punishment for breaking the Hippocratic Oath

Leave a comment

Login

Allowed HTML <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre> Markdown Extra syntax via https://michelf.ca/projects/php-markdown/extra/

Sidebar photo of Bruce Schneier by Joe MacInnis.