Oil and gas production is an extremely volatile business. From exploration and drilling, to transport and delivery, producers are continuously faced with difficult challenges that can mean the difference between profitability and loss if they are not overcome; and as oilfield operations throughout the world have become increasingly complex, the inherent risk that companies have incurred has increased as well.

While this risk has forced producers to take more extreme measures in order to effectively manage their assets, it has been accompanied by a much greater reward. Recent developments in oilfield technology over the past few years have allowed companies to gain access to reserves that were at one time considered inaccessible, and although some of the most promising reserves on the planet still lie miles beneath the surface, the challenges they present can be met with planning, better decision-making and more effective data management.

Managing risk means understanding risk and understanding risk means understanding data. The most prominent obstacles in the oil and gas industry today (particularly in the upstream sector) have come from an inability of producers to quickly and accurately sort through the astronomical volume of unstructured data being gathered from various oilfield operations. By having the ability to consistently interpret this data in real-time, companies can mitigate risk by identifying useful trends, predicting events and making more intelligent decisions regarding their activities.

The benefits associated with effective data management can be seen throughout all phases of the petroleum industry. From the allocation of people and resources, to EHS activities and accounting, it has slowly been woven into the fabric of oil and gas production. Data management has given producers the ability to accurately view their operations from a large-scale perspective and in doing so they can meet the challenges at hand and accurately predict future obstacles with a high rate of success.

Data is no longer a tool; it is an asset and it’s become just as important to a businesses’ overall success as the people interpreting it. In addition to providing scientists and engineers with an accurate depiction of subsurface conditions, effective use of data can help pinpoint drilling locations, predict mechanical failures in equipment before they occur and optimize oilfield operations, all of which result in higher profits, more safety and increased productivity. But along with this promise of increased profitability has come an increased demand to remain environmentally sustainable.

Environmental accountability has become a top priority not only among regulating agencies but also among producers themselves. And with this trend, companies have had to shift a great deal of their attention to preserving the natural ecosystem without sacrificing productivity. As a result, the term “risk” as it pertains to oil and gas has become much more encompassing and while increasing profits has remained a top priority, doing so at the expense of the environment is considered nothing short of failure.

As oilfield operations have advanced and evolved over the past decade, so has the risk associated with them. It has become more complex, more unpredictable and increasingly difficult to avoid with traditional techniques. And just as quickly as producers are discovering this to be true, they are also discovering that the most efficient way to manage this risk is through effective data management and proper allocation of resources. Consequently, the companies that are able to keep pace with Big Data and the challenges it presents are the companies that will remain profitable for many years to come. Poor data management costs major oil & gas producers up to 22% of their annual income

In 2012, Oracle released its report: ‘From Overload to Impact: An Industry Scorecard on Big Data Business Challenges’. They interviewed 333 executives representing North American firms, asking them to define the obstacles and difficulties they encounter in handling the overflow of data within their organizations, and how well they are using that information to drive profit and growth.

The industries covered by the survey were: airlines, communications, consumers’ goods, financial services, healthcare, life science, manufacturing, public sector, retail, utilities, and of course – oil and gas.

The results of the survey show that on average, the volume of data an organisation has to ‘digest’ increased by 86% in the last two years. Most of the companies find this situation difficult to deal with – 60 % of the surveyed executives scored their organisations as having significant gaps in human resources, processes and tools to cope with such an increase. Almost all (97%) interviewed executives said that a change had to be made to improve the situation.

Of all the surveyed managers, 93% believe they are losing revenue as a result of poor data management. Organisations with the annual revenue of £1 billion or more declared they lose about 13% of it as a result of inadequate data management. This can be translated into $130 million in lost opportunity per annum!

For oil and gas sectors this loss is even more significant and can reach up to 22% of the annual revenue! This is the most among all the surveyed industry sectors. This finding confirms that the robust data management systems are crucial for the profitability and market competitiveness of the oil and gas producers.

Most of the organizations (77%) see the opportunity for improvement in industry-specific applications and software. The example of such solution for the oil and gas sector could be information system designed to monitor oil reserves and distribution, helping to meet the current demand.

Among all surveyed oil & gas managers, 74% admitted to use industry-specific software for making strategic decisions, while project management (36%), regulatory compliance (32%) and customer relationship management (26%) were identified as the key areas where with the room for improvement.

The main problems identified within the sector are:

• Business managers cannot get access to the information they need- they have to rely on IT department (32%)
• The organization does not have the right system in place (29%)
• The system in place is not designed to meet the industry specific needs (23%).

Most difficulties could be addressed through:

• Providing business managers with an access to the critical business information
• Improving training to help understand the data, and
• Improving tools so that they can collect more accurate information.

Jeffrey Maskell is CEO & Group Managing Director of London-based Westheimer Energy Consultants