Burglary Detection through Video Analytics

This is interesting:

Some of the scenarios where we have installed video analytics for our clients include:

  • to detect someone walking in an area of their yard (veering off of the main path) that they are not supposed to be;
  • to send an alarm if someone is standing too close to the front of a store window/front door after hours;
  • to alert security guards about someone in a parkade during specific hours;
  • to count the number of people coming into (and out of) a store during the day;

In the case of burglary prevention, getting an early warning about someone trespassing makes a huge difference for our response teams. Now, rather than waiting for a detector in the house to trip, we can receive an alarm signal while a potential burglar is still outside.

Effectiveness is going to be a question of limiting false positives.

Tags: , , ,

Posted on July 14, 2010 at 12:54 PM • 39 Comments

Comments

Dinah • July 14, 2010 1:13 PM

Great idea but you’re spot on RE false positives. I can’t think of the problem of false positive without thinking of car alarms. Car alarms are one of the pieces of everyday tech I despise the most. It combines the abrasive attributes that make alarms effective with what feels like 100% false positives. I’ve never actually seen anyone hear a car alarm and react as though they thought their car was being broken in to.

HJohn • July 14, 2010 1:16 PM

@: “Effectiveness is going to be a question of limiting false positives.”

Definitely, and I expect it to be so for the foreseeable future.

Some of the bullet points seem more logical than others. Bullet 2 and Bullet 3 seem pretty straight forward and effective, it’s good to know someone is standing outside and you may want to check it out.

Bullet 1 and Bullet 4 seem a bit tougher. There are many reasons someone would not be on a designated path, and bullet 4 seems very prone to false positives too. Then again, if effective, it would be good to know that 100 customers appeared to enter and only 99 appeared to leave.

The number of false positives will determine not only accuracy, but also how seriously alarms and warnings are taken. When car alarms go off, is there anyone left that really worries that a car is being broken into?

Christian • July 14, 2010 1:21 PM

I am waiting for an AI system behind any public video camera deployed …
finally perfect total surveilance possible without much cost… may be in another 10 or 20 years..

Narr • July 14, 2010 1:48 PM

I am waiting for an AI system behind any public video camera deployed …

Why? Right now your camera system could alert your phone and send clips + live feed for you to examine and let you decide to call the cops on intruders or the plumber on a water main leak. AI might not ever be able to tell WTF is going on as well as you can, especially if you train yourself by watching the cameras off-and-on during normal situations.

HJohn • July 14, 2010 1:49 PM

@Christian

The obvious plus to AI is it can definitely remove all human emotions and predjudice from systems. It couldn’t care less about race, gender, orientations, the feelings of the subjects, its own reputation, discomfort, or safety. It just does what it’s programmed to do.

This is also the downside of AI. It sometimes will not know what to overlook, and it won’t have a gut to tell it that, while things do logically seem up and up, something just isn’t right.

I remember the movie 2010, sequel to 2001. In 2001, HAL went berserk and tried to kill everyone. In 2010, it was determined why–he was given two sets of rules, one to complete the mission with the humans, and one on how to complete it without the humans should they not survive. When the humans aborted the mission plan 1 was eliminated, and he started plan 2 which required all to be dead. Though science fiction, computers, even artificially intelligent ones, will likely do exactly what you tell them to do–even if you don’t realize it is what you told it to do.

mcb • July 14, 2010 2:00 PM

I try never to use a human to do something a machine can do better or cheaper.

Implicit in the decision to do without analytics is the notion that the officer can watch multiple real time video feeds and remain alert enough to detect suspicious events and fresh enough to tell the difference between false positives and rare actionable event. That level of attention can be maintained for several minutes, but not much more. That is why most video recordings are used to investigate situations after they have happened.

Even at the current level of sophistication analytics can help separate a lot of what might be wheat from most of what is probably chaff. We still need a human to assess the precise nature of exceptional events that trigger the system – most of which will be false positives – but we need not hypnotize the officer in the mean time. In time it may be possible to move toward closer to real time observation and response, perhaps even some behviour prediction. Then I’ll worry about false negatives…

Andy Skelton • July 14, 2010 2:04 PM

The effectiveness of biological intelligence is also a question of filtering out irrelevant details—false positives—from the vast stream of sensory data. We have a wonderful ability to filter data and focus attention without thinking about it. I wonder what portion of our wet CPUs are engaged in this activity all of the time.

Pat Cahalan • July 14, 2010 2:26 PM

The effectiveness is going to be a question of not forgetting about the false negatives, or intentionally injected false positives. False positives aren’t necessarily a major problem.

Let’s say you have N square feet to monitor. You can cover that set S of N square feet with M people. Of course, not every square foot is equal, some subset(s) S’, S”… of the N square feet is of higher value than others. Each one of those subsets might require a greater presence than the remaining subsets. Optimally, of course, you want one capable observer
for each S’, and maybe a floater for the less-critical spots.

Something like this is a hinky-o-meter, it’s effectively adding another observer to all the spots (both S’ and non-S’) in the problem space, albeit one with limited perception capabilities.

A false positive isn’t necessarily a bad thing; if you have to have M people anyway, having one of them drift over to check out grid A19 isn’t necessarily a loss of effectiveness… if the grid they’re leaving isn’t leveraged in their absence.

If you don’t even have M people to begin with (think neighborhood watch programs), but just have the ability to “summon eyes” (again, think neighborhood watch programs), this might not be a bad system even with false positives, since you’re using it to temporarily promote someone to a role that they don’t possess by default.

It’s only a loss of effectiveness if Security Officer Bob discounts his own observation of possible hinkyness to go where the computer tells him to go, or if someone games the system to get everybody to come to grid A19 when the real action is going on in Q139 in 30 seconds.

Sure, enough false positives and the system is ineffective because nobody pays any attention to it, but I don’t think an occasional false positive is as big of a design problem in something like this as it is for many other types of security systems.

max • July 14, 2010 2:26 PM

There must be a large amount of footage from existing surveillance footage available for testing. It would be interesting to feed a bunch of that footage into this system and see how many false positives (and false negatives) it comes up with.

No One • July 14, 2010 2:40 PM

On the subject of car alarms, I wonder what the rules are for taking video with sound inside your own car (where I live and elsewhere, of course) — it would be interesting to hook up a camera (or two, to watch the front seats and rear seats) that calls your smartphone and transmits audio+video of what’s going on when the alarm goes off. This would mean that when an alarm goes off you just have to check your phone rather than run out to your car, like if you live on the third floor and parked around the corner. It could also give you video to review of the thief to help catch and convict him. (I imagine most car thieves don’t wear a mask — it seems like it could bring undue attention.)

I guess the risk is that when your alarm does go off if you don’t go out and check the car manually the thief may have just waited for you to turn it off remotely when you saw no one inside (yet).

mcb • July 14, 2010 4:56 PM

@ No One

Depends on whether your state requires two-party notification of recording. In such states a small sign on the dashboard might suffice…please consult your attorney before implementing your plan.

Otherwise I like the idea of making each car owner their own alarm monitoring company instead of waking up the entire neighborhood with a audible car alarm to which no one ever seems to respond.

bob!! • July 14, 2010 7:25 PM

If you can feed the video to the security company, it would be a really useful tool. In the video, they talked about setting up a video analytic rule which creates an alarm if someone climbs over the fence – if it then feeds the video to the security company and they see that the guy who just climbed over from the neighbor’s yard picked up his football and then jumped back, then they don’t need to call the police, whereas if he’s sitting there trying to pick the lock on the back door, they can, before an alarm on the door triggers. If the alarm company is willing to sort out the false positives before calling the cops, it seems like a net positive to me.

Thinkerer • July 14, 2010 7:30 PM

“Now, rather than waiting for a detector in the house to trip, we can receive an alarm signal while a potential burglar is still outside.”

So the criteria for arrest would be…?

Frankly, I think that this would be too much fun for kids to resist if a bunch of blue-suited goons showed up every time they did something goofy in front of a surveillance camera.

Think of YouTube with a clown act follow-on…

framecrash • July 14, 2010 7:48 PM

For what it’s worth: Last year I caught a burglar in my home with a motion sensor camera feeding images to a NAS box that’s hidden. Cops arrived. The city DA took the case based on my images. Burglar’s now doing 16 years.

I’d guess Bruce would put this under “detection”. See the URL at my name for images of the detection.

NobodySpecial • July 14, 2010 9:49 PM

A group at QMC (univ London) were doing something like this 20years ago.

An expert system watched the CCTV cameras and learned where people normally moved in a space, then could trigger an alarm when somebody went into a previously off limits area.
It was used to set up virtual fences for museums, kids around swimming pools, pedestrians straying onto train lines etc.

Matt from CT • July 14, 2010 11:40 PM

I wonder what the rules are for taking
video with sound inside your own car

Video should be OK in all U.S. jurisdictions.

Sound may run afoul of wiretap laws in certain U.S. states.

Moz • July 15, 2010 2:35 AM

I wonder what the rules are for taking
video with sound inside your own car

Neither is permitted in some parts of the US if a police officer is in front of the camera. The presence of the camera might even be justification for entering the vehicle and disabling the camera.

Karl • July 15, 2010 3:09 AM

There’s a company that recently launched a large ad campaign about their surveillance webcam that has “Homeland Security technology.” Of course, they don’t actually define what that is, so it sounds like a bunch of BS to me. If you dig deeper, it almost sounds like they’re trying to do something similar.

Peter A. • July 15, 2010 4:50 AM

@HJohn: AI […] can definitely remove all human emotions and prejudice

Unless it’s programmed in! 🙂

And, frankly, it could be even reasonable, justified and effective if done according to the outcome of sound statistical analysis over a large body of footage. But I am inclined to think it won’t be prejudice then, even if it may look like one.

Over-simplified example: since most of the burglary cases are commited by men, the system detecting a person straying off the path may assign a lower score to the event if the person is identified as a woman. Is this a prejudice? I’d say no. Of course if this property of the system becomes known and widely deployed, burglars will wear a disguise…

Simon Farnsworth • July 15, 2010 4:54 AM

I do sometimes wonder, given the prevalence of apparent false positives from alarms, whether a simple “reset” remote control that could be given to friends and neighbours would be useful.

For clarity – I assume that the basic working model of an alarm is that you have one or more sensors that indicate a bad condition. When the sensors trigger, they’re fed into a latch, which drives the alarm state. The reset control I envisage would simply reset the latch; if the sensors are still seeing a bad condition, the alarm will continue sounding even after the reset control has been used.

There’s a security tradeoff here, which I think is advantageous to the legitimate owner.

On the one hand, it is in theory possible for someone with the reset control to keep resetting the alarm until they’ve worked out how to avoid retriggering it – ameliorated by the fact that the alarm continues to sound until you stop retriggering it.

On the other hand, if I can stop your alarm disturbing me, I have an incentive to go and investigate, and if it’s a genuine alarm, I’ve just caught someone in the act, and I can phone the police for you.

BF Skinner • July 15, 2010 6:36 AM

@mcb “…requires two-party notification of recording”

Actually I don’t think many if any states have two party laws regarding image taping. The incidents in Boston revolve around the wiretap law and are being misinterpreted for ANY audio recording of speech conversation regardless of communication channel.

The Illinois incident did however make refernce to “electronic survellience” laws.

@Moz “…neither is allowed if a police officer…”
Would you cite the states and statues?

See there’s what the police do and believe and then there’s the law. Police, some, believe they should get the bad guy, even if it leads to a shoot out, even if the shootout is with an unarmed citizen, even if it means planting a weapon on the formerly unshot unarmed citizen. They got the bad guy. If they weren’t guilty of the particular crime the police were investigating then they were probably guilty of something else.

I know there was talk about laws to prevent photographing of police but I never heard them pass.

Clive Robinson • July 15, 2010 7:19 AM

@ NobbodySpecial,

Hmm QMC comment and UK time zone posting, guess you might be on this side of the puddle 😉

Don’t know if you know but some bods at Kingston University did something similar that was tested by TfL on the LU looking for buskers beggers and other naredowells who might just be casing the joint for another 7/7 or similar.

From what I read (in press blurb so take a pinch of salt) the system had the ability to track movment in crowds and see if somebody had a pattern to their movments that was disimilar to others around them and draw an operators attention to them. If true it would be more than most humans could do who usually cannot track more than two individuals at any one time.

On another issue there was a “competition” held every four years to evaluate facial recognition software. It is about four years since the last one and if I remember correctly the improvment in that one was about ten times the one four years before…

Thus if the technology has progressed ten times again then it may now be possible to track people in crowds based on their daily routien being changed as well as odd behaviour (thus hopefully detecting “jumped / fell / pushed’s” before they jump under the wheels).

HJohn • July 15, 2010 8:47 AM

@Peter A: “Unless it’s programmed in! 🙂

it could be even reasonable, justified and effective

since most of the burglary cases are commited by men”

I don’t disagree, but programming it in would also be a lawsuit waiting to happen. Try it on any characteristic other than gender based on crime rates and it would probably not end well….

That said, while I would certainly be more concerned about a 6’5″ man than a 5’2″ woman in regards to crime, I can see where any exception could be used to circumvent.

But it does make for interesting discussion.

John Ribbler • July 15, 2010 9:35 AM

Interesting, but the FutureSentry robotic guard performs those functions and many more, at much lower cost.

Analytic cameras can only track what comes into the view of the camera. To cover diverse facilities and terrains, requires many cameras. Whereas, one robotic guard — guided by remote sensors — can protect a much larger area.

Most importantly, the robotic guard actually gets involved with and intimidates potential intruders. It doesn’t just take pictures and set off alarms.

Check it out: http://futuresentry.com

paul • July 15, 2010 10:01 AM

Note that what this system does, in effect, is to take advantage of the current price differential between video-plus-computer and a bunch of proximity sensors connected to a much cheaper CPU. Lots and lots of the potential false positives are about the fact that one or two simply-placed cameras can’t see the crucial details that would differentiate between “innocent” and “guilty” behavior.

It’s all about rethinking what things you want to alarm on, and what levels of alarm status you have available to you.

Btw, I wouldn’t want to rely on a person count from a single camera viewpoint. Way too much chance of shadowing, and other sensors could do this better.

Geek Prophet • July 15, 2010 11:05 AM

@HJohn

True, but “programmed in” can have more than one meaning.

For example, I could build a system, give it the best detection systems known, then give it the ability to program itself heuristically. Show it a lot of video of normal days and a lot of video of what happened shortly before crimes, and let it look for patterns and learn to watch for them.

Even if the video is selected completely without prejudice, if most crimes in the areas the video is from are performed by people with certain descriptive characteristics, the software could program itself to watch people of certain ethnic types, genders, or clothing styles with any racism whatsoever.

Of course, if you wanted to bias it, you could bias the video selections, and you could probably get away with it if you were careful. Or if you just were biased yourself, and more likely to overlook minor incidents that involved people you weren’t biased against, again, you might influence it.

It is possible to create “racist” algorithms just by how we choose the input we give it during “training”.

y2me • July 15, 2010 11:07 AM

MOZ those laws against filming police are made so the impunity and corruption can continue with less protest. However, cell phones have video and it can easily be posted on the web. the public will not be so easily fooled anymore by the official lies.
criminals also would like the cameras to be removed.
If a store can surveil its sidewalk, you car should have the same rights, perhaps you could incorporate your car so the supreme court could be on your side. the hate peoples rights and diminish them every time they meet but they are giveing corporations rights that are better protected than your individual rights.

Michael Jagger • July 15, 2010 12:01 PM

The post that Bruce links to is mine.

I think that it’s important to clarify that we use this technology mostly in residential settings. Our company provides a guaranteed five minute response to client’s alarms and we monitor/respond to about 6,000 homes in Vancouver. We respond to 100% of our alarms ourselves.

We use video analytics to help direct our response teams to potential burglaries. Any time an alarm is received, our operators in our central monitoring station take a quick look to see if anything looks suspicious. If there are any notes on file (ie. that the client is out of town and noone should be at the home). Typically, the analytics are enabled when the main alarm has been set… so we do not get alarms for the owner wandering around their own property. Further, the field of view for any analytics -enabled cameras are as narrow as possible to avoid false alarms.

In the event that we see that someone has jumped the fence, or is wandering in an area of the property that we would consider suspicious, our response teams attend and are able to talk to the person.

Granted, charges are almost never laid for trespassing, but the actual burglar gets a strong message that this house is not a good place to be skulking around. Given that the jail time served by most burglars is so short, we’ve found that the preemptive response offers a much stronger deterrent to crime: at least crime at that particular address.

In my experience, the value that our response teams can offer in a situation where the are responding to someone who is on the property, but not expected to be there, is superior to waiting for an actual alarm to be tripped on the house where we have to respond to an actual burglary in progress.

No One • July 15, 2010 12:24 PM

Thank you, Michael, for adding this info — so basically your company acts like a neighborhood watch, but professionally instead of doing it all amateur-hour.

Do you know if your company has considered group discounts for neighborhoods? Kind of like the idea of herd immunity — you could, by offering twice the coverage at half the price reduce crime (and thus your costs) in a win-win manner?

Michael Jagger • July 15, 2010 12:39 PM

@ No One

That’s exactly what we do. However, in addition to monitoring & responding to the alarms, we also provide Homewatch services. When you go out of town for the weekend, we’ll come by and water your plants, feed your cat, walk your dog or fill your fridge for your return home. We also drive clients to the airport, pick them up, etc. Basically, we will take care of everything while you’re gone.

We work for several different neighbourhood groups… with varying degrees of organization and coordination behind them.

In most cases, the neighbours pool their resources and contract with us to provide regular patrols… given that our billing is based upon actual time spent, the more neighbours involved, the cheaper the individual cost. Some have elected to set-up dedicated 12 or 24 hour patrols that never leave.

In terms of the geographic areas that we serve, in order to provide a guaranteed five minute response, we need to keep them as small as possible. This results in much higher security for our clients in that higher client density is both good for our business and better for our clients. There is certainly an incentive for clients to spend their monthly security fees with us than with a company that both does not respond, nor have any local presence.

Ross • July 16, 2010 12:37 PM

@Hjohn “Bullet 2 and Bullet 3 seem pretty straight forward and effective, it’s good to know someone is standing outside and you may want to check it out.”

I’m not really convinced about #2 — I think that would trigger false positives constantly. Lots of people look in store front windows to window shop when the store is closed. Particularly if the lights are dim, that encourages people to lean in close and/or put their hand up to the window to attempt to see inside better. I’m not much of a window shopper, but I’ve still done that often enough.

For that matter, I can think of plenty of times I’ve tried to walk in to an apparently open store only to find the front door locked. At which point I stand there trying to find the hours to figure out if the store is closed or I’m just doing something wrong with the handle; or searching for something that will tell me when the store will be open. Again pretty normal behavior.

I’m trying and struggling to come up with many scenarios where harm comes to a storefront based on people standing too close and not doing anything else.

anonymouscat • July 18, 2010 5:42 AM

Excellent! With a little bit of tweaking I am sure it can also be used to ‘detect’ cops. Great stuff for criminals!

SnallaBolaget • July 19, 2010 8:59 AM

Like NobodySpecial said a way up there, this really isn’t anything new. All of the systems I’ve worked with have had some form of this as an option – some worked better than others, of course, but all of them have had a version of this.
Detectors and camera feeds have been analysed for a long time, both by humans and machines, and though the “arrest criteria” aren’t there for something that hasn’t happened yet, as on commenter also asked about, a well known scientist once said that an ounce of prevention is worth a pound of cure. It’s better to prevent it, that means, than to fix it after it happened.
Now, if the alarm goes off because some Joe Blow has been outside the window too long, pondering whether to throw that brick or not, then you can’t arrest the guy, but if the appearence of a guard company car, or the cops, will make the guy run off, then you’re spared the trouble of replacing the windown and your stuff, and the burglar will target someone else instead.

Interesting, yes. New? No.

SnallaBolaget • July 19, 2010 9:01 AM

@anonymouscats;
Why would cops have any interest in hiding? Especially if they’re in uniform? If you teach the thing to recognize cops in uniform, that’s one thing – how will you teach it to spot a plain-clothes officer, a detective or an undercover vehicle?

I’m just asking.

SnallaBolaget • July 19, 2010 9:05 AM

@Ross;
I agree with you. This is better used as, for example, the “invisible fence” someone mentioned above, where the system detects abnormality in movement patterns within a specified area. Standing still too long can be a sign that someone is thinking about doing something fishy, but it would absolutely generate a lot of FPs.

In any case, there are already a lot of detectors that will trigger only on very specific areas. We had some installed in a facility where I worked, and upon testing them found that their margin of error was in the half-inch territory.
Inexpensive, and does the same job, basically.

Keith Anselm • July 19, 2010 9:48 AM

I’m reminded of the times I was a false positive.

On one occasion, I walked to the convenience store and arrived late. I wanted to know what the hours were for future reference, and discovered it had closed an hour earlier. I must have been standing too close to the door – a passing cop stopped me and searched because “I looked like a potential robber.” I agreed to the search only because I was worried about him potentially planting evidence on me.

On another night, I parked on the far side of a plaza from the store I was trying to reach. A cop pulled me aside and informed me that “someone had seen me looking in windows for thing to steal.”

These were obnoxious, but I think it could be much worse if humans appeared later in the decision tree.

Alex • July 19, 2010 11:15 AM

Intelligent camera’s are a waste of money… cyborg implants watching you 24-7 are the thing to be in, say, 20 years from now. Far better ROI.

Hmm.. what’s with the black helicopter over there?… :-S

Leave a comment

Login

Allowed HTML <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre> Markdown Extra syntax via https://michelf.ca/projects/php-markdown/extra/

Sidebar photo of Bruce Schneier by Joe MacInnis.