Posted on 09/20/2010 8:52:32 AM PDT by RogerFGay
Big deal. Android applications are still Java applications.
“Please note that the NDK does not enable you to develop native-only applications. Android’s primary runtime remains the Dalvik virtual machine.”
“The NDK will not benefit most applications.”
“...using native code does not result in an automatic performance increase, but does always increase application complexity.”
I’d have a really big problem with using learning/adaptive software (did someone say AI?) in safety-critical aerospace applications because such software deliberately fudges the boundaries of what it can and cannot do. That’s where learning occurs - at the edges of the envelope.
But, the edges of the envelope in aero software gets people killed.
I’m very skeptical, but not for the “crazy” reason. I distrust the “unexpected” solutions, too!
I’m in the certification side of aero software, BTW.
I was booked to give a talk at AUVSI’s Unmanned Systems Europe 2009 on that very subject; engineering with learning systems for applications that require rigid quality standards. We had critical events at the company then, so I couldn’t make it. I may yet write the paper though. It’s not so mysterious and difficult - but I’m hiding the reason ... for now. (Drum rolls, marketing hype, raising expectations, music plays, curtain slowly opens ...)
... This paper describes the new software architecture and discusses the potential of application. The current implementation can be installed on a wide range of autonomous systems, automatically locates sensors and actuators and builds its own system specific control programs. Local environment simulations are constructed from sensor input and used as a “robot's imagination” to adapt and solve problems. System behaviour can be extended by training in new environments and providing new challenges, by installing new “fitness functions” to drive learning, and by integrating application specific components developed “by hand.” Adaptation and field creation of new behaviour can be limited to accommodate various requirement levels for testing before use, from exploratory research and experimental development to the most rigid field-ready product quality control standards. We also expect the system to reduce development time and cost.
The issue comes down to meeting regulations, where the system or equipment is required to (1) be designed to properly perform its intended function under all foreseeable operating conditions and (2) be safe to a specified probability, based on the potential effects of failures that the system can contribute to.
Because of the autonomous learning of the AI system, the specific response to a given set of operating conditions cannot be identified before hand.
The current (and immediate future) answer to the second part of the regs is the application of a process-based discipline for developing software. Allowing autonomous self-modification by the software itself does not support this approach. The cert authorities will allow alternative methods, but are (very) unlikely to allow autonomous self-modification of software programs in the near future.
Part of the reason is the outcome of a previous experiment of AI, where a neural net was used to analyze satellite photos for hidden armor (tanks, etc.). The neural net was very successful on its training data - mostly from Germany.
But the net’s success plummeted when it was shown satellite photos of the desert.
They found out that the neural net had settled on cpunting the number of leaves/leafy trees it could see as a predictor of the presence of camouflaged armor - which doesn’t work in the desert.
The process of decision making, when civilian lives are on the line, is not yet ready to be delegated to leaf-counting programs! :)
Have a great day!
Part of my standard talk is that “it’s not yet time to fire all the engineers.” Who turned the sucker loose in the desert when it was only trained in the Black Forest? I’d a seen that one coming a thousand miles away.
I guess I’d have to ask - why wasn’t the experiment carried out by experts?
BTW: I could never get all that excited about neural nets.
It was.
The moral of the story is that the margin of error for software running some of these systems is ten to the minus ninth power - extreme improbability.
Allowing a computer to teach itself can’t reach that level of certitude.
Even the smallest glitch would result in industry-killing liability lawsuits the likes of which we’ve never seen.
The risk is not worth the reward.
If I get the paper written, I’m sure I’ll blog about it. The issues you raise are terribly obvious. Can’t think of any reason I’d imagine that someone in the field wouldn’t understand them.
That’s because the “field” was a sandbox back when this experiment was performed. It was one of the first successful neural nets, and it had to be handled by people who knew the field it was dabbling in - satellite photo reconnaissance.
Hindsight is 20-20 in most instances, but sometimes the people in charge aren’t looking at the history one would like them two.
For many applications, I think it’s fine to use self-teaching software. At this point, I don’t agree that the aero field is ready to take the risk.
Have a great evening - NCIS is coming on, so I’m going offline. (A man has to have a FEW vices, right?!? ;-P)
There is a great deal of benefit to using automated tests, they will help insure that positive flows through the code's logic work, and that common error handing routines are sufficient. I have over a decade of OO automated tool experience so I am not panning well constructed automation, but I guarantee you I can break any code tested only in this manner.
In my humble opinion, the path to achieving what you have outlined is:
- Components need to be simplified, documented, thoroughly tested, and rock solid.
- Developers need to be penalized for reinventing the wheel, unless it is truly a better wheel, and then the old wheel should be thrown out.
- QA needs to be brought into the process at inception, not after development has already started.
- QA cycles should not be compressed in order to make up for development overruns (yes I know that I'm dreaming here).
- Project management needs to play a much more active role in projects while they are in flight and learn to close them out when complete to the original scope.
- All major development methodologies are valid and sound if the rules are followed, however most of the time the rules are bent or broken.
NCIS has been one of my favorites. Can’t keep my interest up in re-runs forever though (which is what we get were I am).
There are certain basic scientific rules that aren’t violated when you switch the form of implementation. Pattern recognition is one of those things I focused on as a student; and even though I did different things in the real world, there were plenty of bits of scientific wisdom from that part of my education that served me well.
Just to bring you up-to-date, there are now techniques using genetic programming, sometimes combined with neural networks that do quite well.
Also, consider the fact that you - as a human - automatically notice things in your peripheral vision that grab your attention: a highly effective sort of early warning system. You don’t have to be staring right at something that should get your attention and thinking consciously about it. You don’t have to look at it clearly and for a long time, studying the many subtleties to have your early warning (sort of, it doesn’t only apply to bad things) alarm tripped.
This has led to some interesting experiments in improving recognition by actually reducing - filtering - the information being processed. Counter-intuitive; but it’s improved reliability - and it’s faster of course.
Yes, I agree with much of what you say. I don’t see why that would be a problem with the suggestions I’ve made. In fact, I’ve pointed to the fact that keeping up with modern technology, modern project process has become more agile. This means that project participants should be involved in the flow of work - must less like the olden days when project emphasis would shift from one group to another in large phases.
The one thing about your comments that leaves me a little cold, is the way you want QA people to monopolize testing. If engineers do no testing, then they’ll end up shipping a lot of stuff that doesn’t work to QA. No point in that. And software systems need to end up doing what they’re intended to do - and for many best efforts that involves experts and specialists in the application area (and often end use customers).
Each has a particular role within the quality assurance process.
I want a gold star for this comment: Quality is everyone’s concern!
Last night was the season premier of NCIS.
I understand that I am not on the cutting edge of self-teaching software, but I’m afraid I will remain skeptical on its utility in aerospace.
The very concept of “develop (ad hoc), decouple, and thoroughly test” is well below the minimum requirements for process as they are currently written. I don’t see that changing any time soon - and I am on the committe working on the next generation of guidance on the subject.
To my knowledge, there have been very few aircraft incidents caused by software. I think I heard of a single one, but human memory is frail. To radically change what works - what makes for a high degree of safety in the fielded systems - doesn’t make all that much sense, especially when doing so requires adoption of unproven technology.
And, before you protest my choice of the word “unproven”, consider that the aerospace industry is just now coming to grips with OOT. We’re behind the times, but people stay alive...
Have a great day, Roger.
Actually, they now have systems designed to detect wind shear.
But, one cannot consider the lack of a system (including software) to perform a function as having the software contribute to a failure caused by the lack of that system.
That’s kind of like saying “I turned into the wrong driveway and hit a tree I didn’t have”.
Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.