See Assignar in Action

Opportunities in Computer Vision and Artificial Intelligence for Assignar

Introduction

Demand for development projects and new buildings reached a record level in Australia. Public orders, increased demand for new housing and energy renovation are driving the construction industry to rationalise. But this industry is a poor performer on several levels: concerning the environment, 40% of the volumes of waste generated and greenhouse gas emissions are attributed to it. And in terms of safety, the construction industry remains over the years the industry with the highest occupational accident rate.

The construction industry has a lot of potential for improvement to digitalise its activity and optimise its impact on these various aspects. 62% of construction companies still use manual reporting systems. Video recognition is one of the technologies that provide concrete answers to control operations and develop security systems.

3 key figures of the sector:

  1. 50% of construction companies in the world have not defined technology roadmaps.
  2. 1BN Euros is what acts of vandalism cost the construction industry, i. e. 1% of the turnover of the construction sector.
  3. 88K accidents were recorded on construction sites in France in 2016.

Safety On Construction Sites

Promoting safety and tackling strenuous working conditions are essential issues for the construction industry, which is the most accident-prone sector. There were hundreds of thousands of accidents on construction sites in the world and thousands of deaths. The construction industry is seeking to exploit new digital technologies to move from a curative to a preventive operating mode.

Solutions

  • Detection of the non-use of PPE – Alert your foremen in real time when a worker is not wearing his personal protective equipment (PPE).
  • Detection of dangerous areas to demarcate – Set up an alarm system when a person crosses an imaginary line of protection.
  • Speed control of wheeled vehicles – Control the movements of construction machinery to avoid collisions, especially when reversing.
  • Detection of a dangerous proximity between a worker and a machine – Trigger an alert when the positioning of a machine presents a danger.
  • Inspection of inaccessible structures – Inspect your infrastructure with drones or robots and automatically detect signs of deterioration or anomalies.
  • Detection of a potential collision between a crane and a worker – Be alerted to the risk of collision when a crane handling large objects is in the vicinity of workers.

Advantages

  • Recent studies have shown that prevention improves economic performance in most cases: “For every $1 invested, the average return is $2.19.”
  • Improve your employees’ work environment with optimal prevention
  • Avoid delivery delays
  • Reduce costs related to work accidents

Security on construction sites

Acts of vandalism on construction sites are globally estimated at a minimum of 1% of the turnover of the construction sector, i.e. more than one billion dollars. These include theft (construction machinery and vehicles with their contents, technical materials and equipment, tools), various types of damage, arson and squatting in premises awaiting reception.

Objectives

  • Prevent the theft of equipment and materials on construction sites
  • Prevent unwanted intrusions

Solutions

  • Intrusion detection on construction sites – Be alerted when a person enters the site at closing times, of a masked person or a person with suspicious behaviour.
  • Access control on site – Secure your site by controlling access authorisations as well as the unauthorised use of tools or equipment when employees enter and leave the site.

Advantages

  • Avoid delays related to equipment theft or damage
  • Secure your sites 24 hours a day at lower costs
  • Reduce the costs associated with the purchase of stolen equipment and materials

Digitise the industry

Challenge

The construction sector is one of the least digitised. The progress of the work is monitored manually, often causing errors and confusion and consequently delays in delivery. One of the solutions identified to remedy this is the development of Building Information Modelling (BIM), a kind of digital model of the construction site, which is essential for the digital transformation of the sector. Video recognition allows you to extract information about completed steps and stocks.

Objectives

  • Go from manual monitoring of site progress to digital solutions
  • Know the stock situation in real time
  • Eliminate non-value-added activities

Solutions

  • Site mapping by video analysis – Measure in real time the progress of the construction site.
  • Automatic monitoring of site activities – Report on the tasks performed by each employee with a single click.
  • Stock management – Monitor in real time the stock status of your raw materials.
  • People counting – Be informed in real time of the dynamism of your site by counting the number of workers present.

Advantages

  • Reduce non-value-added activities and optimise your site management
  • Increase the productivity of your workforce
  • Reduce construction delays due to stock issues

Conclusion

Computer Vision functionality has traditionally been very expensive to implement and experiment with. At Assignar, we are fortunate to have been architected on top of AWS platform. AWS allows us to leverage technologies such as AWS Rekognition to quickly develop and deploy computer vision applications at scale with a fraction of the cost in comparison to some of the traditional architectures. We have built an initial POC that analyses all images submitted from the field and detects objects inside them. Once the metadata is extracted from images, further analyses and workflows can be triggered. This is our architecture diagram and the snippet of code that does the bulk of the work:

Rekognition Architecture Diagram

Rekognition Architecture Diagram

'use strict';

console.log('Loading function')

const aws = require('aws-sdk')
const rekognition = new aws.Rekognition()
const doc = require('dynamodb-doc')
const dynamo = new doc.DynamoDB()

/**
 * 
 * @param {*} event 
 * @param {*} context 
 * @param {*} callback 
 * This lambda function gets triggered by ObjectCreated event in the Assets S3 bucket. 
 * The object is then run through an AI process that detects labels and text in each image and stores it in DynamoDB for search and analysis.
 */
exports.detectLabels = (event, context, callback) => {
    console.log('Received event:', JSON.stringify(event, null, 2))

    // Get the object from the event and show its content type
    const s3Bucket = event.Records[0].s3.bucket.name
    const s3Key = decodeURIComponent(event.Records[0].s3.object.key.replace(/\+/g, ' '))
    const pathInfo = s3Key.split("/")
    const TenantId = pathInfo[1]

    const labelParams = {
      Image: {
        S3Object: {
          Bucket: s3Bucket, 
          Name: s3Key
        }
      }, 
      MaxLabels: 50,
      MinConfidence: 50
    }

    rekognition.detectLabels(labelParams).promise()
    .then((data, err) => {
      if (data) {
        return data // successful response
      } else {
        context.fail(err) // an error occurred
      }
    })
    .then((labels) => {
      let Labels = []
      let Confidence = []
      // let's flatten some important data for easier searching
      labels.Labels.forEach((label, index) => {
        Labels.push(label.Name.toLowerCase())
        Confidence.push(label.Confidence)
      })

      const DateStamp = Math.floor(new Date())
      const MetaData = labels
      const ItemType = 'object'
      const SearchText = Labels.toString()
      const dynamoParams = {
        TableName: `${process.env.STAGE}ImageLabels`,
        Item: {
          TenantId,
          s3Bucket,
          s3Key,
          MetaData,
          Labels,
          Confidence,
          SearchText,
          ItemType,
          DateStamp
        }
      }
      return dynamo.putItem(dynamoParams).promise()
    })
    .then(() => {
      detectText(TenantId, labelParams, context)
    })
}

function detectText(TenantId, labelParams, context) {
  // Get the object from the event and show its content type
  const s3Bucket = labelParams.Image.S3Object.Bucket
  const s3Key = labelParams.Image.S3Object.Name
  delete labelParams.MaxLabels
  delete labelParams.MinConfidence

  rekognition.detectText(labelParams).promise()
  .then((textData) => {
    if (textData.TextDetections.length) {
      let Labels = []
      let Confidence = []
      // let's flatten some important data for easier searching
      textData.TextDetections.forEach((text, index) => {
        Labels.push(text.DetectedText.toLowerCase())
        Confidence.push(text.Confidence)
      })

      const MetaData = textData
      const DateStamp = Math.floor(new Date())
      const ItemType = 'text'
      const SearchText = Labels.toString()
      const dynamoParams = {
          TableName: `${process.env.STAGE}ImageLabels`,
          Item: {
              TenantId,
              s3Bucket,
              s3Key,
              MetaData,
              Labels,
              Confidence,
              SearchText,
              ItemType,
              DateStamp
          }
      }
      console.log('text found in image, storing in dynamo')
      return dynamo.putItem(dynamoParams).promise()
    } else {
      console.log('No meaningful text found in image, finishing up.')
      return true
    }
  })
  .then((result) => {
    context.succeed(result)
  })
}

 

Customer-focused technologist with years of experience in developing and architecting world-class web solutions and products. I see technology as a tool to drive positive outcomes for customer businesses and to make the world a better place. A life-long student rather than a person with 1 year of actual experience that has been repeated 10 or 20 times. I love hands-on work as well as leadership, strategic thinking, scaling systems, teams and giving back to the community. My experience covers a wide range of industries, such as digital agencies, education, media, telecommunications, government, and most-recently - construction. Very grateful for the life I have had and the people I am surrounded with. Co-Founder and the CTO of Assignar.

Related Articles

Share This