Wednesday, September 21, 2016

Improving Safety in Surface Mining with Geppetto

Image: Wikipedia
Surface mining is a modern economic necessity and, all controversy aside, it's not going anywhere soon. One day, the process of removing minerals and ore from the earth's crust may be performed by drones and remote operated machines but until that day, men and women will be working in the high-risk, hazard-wrought environment that is open-pit mining.

The industry does a great many things to mitigate the potential dangers these brave individuals face everyday.  For example, I once saw a really cool piece of surveying tech that automates some of these tasks. Two jumped out at me in particular: slope fracturing measurement and equipment wear detection.

Slope fracturing describes the stability of a slope based on the arrangement and size of the particles in it.  Mines must monitor the degree of slope fracturing in order to prevent dangerous rock slides.  What I saw was a hand-held device that would process, on-site, images of the slope using computer vision techniques and special analysis software to estimate, at a glance, some metric of stability.

Image: Wikipedia
Another important task that this device tackled was measuring the degree to which wear components of excavating machinery had deteriorated.  Operating heavy machinery is dangerous enough in ideal circumstances, but as the teeth on the scoop of an excavator begin to wear, it has to work harder to do its job.  A weak everyday metaphor of the ensuing situation is using a knife in a kitchen.  The duller your blade becomes, the more you compensate to get the job done.  Suddeny, you're pressing down with all your strength to cut through a potato, for example, when the knife slips and cuts your hand badly.  Damage to your knife blade, like chips and burrs, can cause their own problems as well.

Similarly, as the teeth of an excavator scoop wear down and its work begins to require more force, the operator will inadvertently begin to compensate.  Eventually, like the slipping knife, the machinery may experience a severe and life-threatening mechanical failure. 

Now imagine an industrial kitchen with hundreds of knives to maintain.  It would be good to be able to look at whole batches of these knives and immediately be able to tell which ones need to be sharpened or replaced.  With the replaceable teeth on mining equipment, there is a standard level of degradation permitted before they are replaced.  Without assistive technology, each tooth of each machine would have to be inspected.  The device I saw allowed an individual to audit an entire array of parts at once and tag those in need of replacement, again using an on-board camera and computer vision software.

Both of these processes involve the physical presence of an expert to manipulate the device and interpret the data.  To put this in the context of the industrial kitchen, let's equate slope fracturing to the state of decay of food. Under this system, the chef must go through every fridge, freezer and storage room to ensure every steak is fresh, every leek is green and every expiry date hasn't passed.  Then, he has to go around testing every knife to make sure it is in good repair.  This protects the health and safety of his employees and customers.  How do we make his job easier?  What if each cook was knowledgeable enough to make the micromanagement decisions of which head of lettuce was rotten and which blade was dull?

Okay, so this is sort of already the case in kitchens, but maybe we can use technology to make this happen in the mining industry.

Here's my caveat:  I know very little about open-pit mining.  What I know is that mining engineers need to collect slope fracturing data and measure the state of degradation of wear components on mining machinery.  And I understand that this data can be used to improve safety and reduce operating costs.

Designing a Mezzanine Board

So how do we automate an early-warning system for landslides and machine wear in the context of surface mining?  So we have two computer vision tasks, one of which takes proximity as an input.  Other potentially important variables are geographic location and heading.  We also need some powerful brains to process this data.  Oh yeah, and we need to get the data from the field to an off-site computer for analysis.  Oh and let's keep costs and power consumption to a minimum, shall we?

The Big Idea

What if we could equip every excavator, bulldozer and dump truck with the means to relay slope fracturing and mechanical wear data off-site in real time, providing analysts with the means to monitor the safety of the equipment and terrain? Then they could dispatch crews to deal with these issues before they become problems.  Let's slap a little device in or on the cab of the machines at the mine, attach a few devices and antennas to it and give it the ability to perform the two tasks I described above.

It'll need two views, one facing forward and one in view of the wear components of the machine when it's at rest.  Since these are unlikely to be the same, we'll need two cameras.  It needs some method of measuring distance.  IR and ultrasonic methods do not have the range needed for the given application so something like LIDAR would be useful.  Positional and directional data are also important for both tasks so some kind of GPS and digital compass will be needed.

Beyond that, we just need a way of processing and transmitting the data for analysis.

So here's most of our hardware requirements:

  • 8MP+ camera  x2
  • LIDAR rangefinder
  • GPS
  • magnetometer
  • LTE modem
  • compute device
Hey no problem.  We're going to use my favourite tool: GEPPETTO D2O.

Compute Device

Geppetto has an ever-expanding list of connectors for COMs.  Recently, we've added 96Boards, TechNexion PICO SOMs, and the new Intel® Joule™ compute module to that list.  What we need is a CM that supports 2 HD cameras.  Coincidentally, the 96Boards mezzanine header does. Specifically, the Dragonboard 410c and HiKey support 2 MIPI CSI-2 cameras (one 4-lane and one 2-lane).  Also, 96Boards are SBCs so they come equipped with WiFi, USB ports, HDMI, and USB-OTG, and include a bunch of low-speed communication buses, such as I2C.  These specs satisfy all of our hardware needs.



Garmin sells a module called the LIDAR-Lite for a reasonable price (~$150 USD) and I'm sure there are similar products from other vendors out there.  It can communicate either over I2C or PWM and is good up to 40 m (+/- 2.5cm).  This would do nicely for our system.


Geppetto has a convenient 5-pin header for Gumstix's Pre-GO and Pre-GO PPP GPS modules, which I have discussed at length in previous posts and is easy to implement in software.  That settles that.


We could use a compass to provide information as to which slope the device is analysing.  If we know where we are and we can tell which way we are facing, we should be able to discern what feature of the terrain we are looking at. Add Geppetto's 9-axis IMU and you get  a gyro and accelerometer as well.

LTE Modem

Geppetto provides a connector for NimbeLink® Skywire™ 4G LTE modems.  50 Mbps upstream is more than enough to transmit the analytical data from the board, even if you want to transmit a live video feed.

My Board: FracJaw

Well that's it.    all that remains is to slap that on a board and click "Order".  I did some playing around and this is what I came up with:

This took me about 30 minutes to come up with. It has my two CSI-2 camera headers, an I2C header for the LIDAR, an LTE modem connector, GPS, and an RTC.  And it all fits on a 13x5cm board., so about the size of your smartphone.

I've saved my board design and made it public in the community tab.  Or you can jump directly to it here.

No comments:

Post a Comment