Tackling Illegal Content Online: European Commission Weighs In With Recommendations

Are online platforms responsible for illegal third party content published on their networks? Many would unequivocally say yes and recent attempts to go after Facebook for its alleged role in facilitating terrorism is a vivid testament to that. In recent months, calls urging tech giants to do more to police their space grew louder internationally as well as in major European capitals. The most stringent ones came from Berlin. According to a new German law that came into force this month, social media companies now have to remove hate speech within 24 hours (in straightforward cases) or face astronomical fines reaching 50 million euros. Viewed in this context, a set of guidelines for online platforms issued by the European Commission last week was anything but unexpected, and the way they were presented suggests the EU is willing to go beyond mere rhetoric and pass legislation should those on the receiving end of the message fail to comply. So what are platforms being asked to do? The request essentially boils down to better detection, prevention and faster removal of illegal content, coupled with greater transparency. In each of these areas the Commission highlighted specific measures it wants online platforms to implement.

Trusted flaggers

Trusted flaggers are widely used by Google to weed out inappropriate content on its services, in particular YouTube. Individuals and organisations can apply to become trusted reporters responsible for flagging content that violates YouTube’s Community Rules. Once flagged, the content is reviewed internally by trained teams who decide whether a reported video should be removed. Google claims trusted flaggers are accurate 90% of the time, or three times more than an average user. The likes of Facebook and Twitter also rely on trusted flaggers but unlike Google, which openly describes its program on the website, these companies hardly share any details with the public. Small bits of information are provided by the Commission and various third parties, but intermediaries themselves are being surprisingly tight-lipped on the matter. More transparency in the way Facebook and Twitter operate their trusted reporter programs is therefore needed, not only to help prospective adopters but also to make their own actions (takedowns) more legitimate.

Notification channels

Besides trusted reporters, the Commission encourages platforms to “establish easily accessible mechanisms [for] users to flag illegal content.” Without specifics one can only speculate as to what they are since many mediums are available for users to report violations. Intuitively, these probably include online forms, flagging buttons and emails although many intermediaries also accept notifications by post. It is not uncommon for intermediaries to have two or even three channels in place. Amazon is a case in point. It provides a form for reporting trademark and copyright infringements. Then each posting and comment can be flagged as inappropriate by logged in users and there is also a postal address for reporting defamatory content.

Historically, email notices were dominant up until 2010, but since 2011 their relative numbers have gradually decreased, giving way to web forms that displaced all other notice formats. Where forms are provided, different types of illegal content can be reported using a single form or several ones. For example, XS4All, a Dutch internet service provider, offers one form to report child pornography, discrimination, libel, slander, IP infringements, unfair competition, violation of trade secrets, spam, malware and criminal offences. By contrast, Twitter has a separate form for each of the following: abusive or harassing behaviour, impersonation, trademark infringement, counterfeit goods, copyright infringement, privacy, private information, spam, self-harm, Ads and components of a Moment. The type and number of notification channels has to be an individual choice of each platform – that goes without saying – but if effective monitoring is a priority then intermediaries should opt for channels that are most conducive to it, such as online forms and flagging buttons.

Automatic detection technologies

The different channels described above tend to involve “manual” work on the part of both senders and receivers (i.e. intermediaries). But a lot of online content is identified and, if necessary, blocked by automatic filtering systems, and it is one area where the Commission wants to see more progress qua investment. Those of us with experience uploading files to YouTube will know what “fingerprinting” means in practice. The process works by scanning your files and matching them against a database of copyrighted material. If there is a match, what happens next depends on rightsholders’ preference; they can choose to block, monetise or track your video.

For violations like child sexual exploitation a similar approach called hashing is used. PhotoDNA developed by Microsoft became somewhat of an industry standard, with major tech giants, among them Facebook, Google and Twitter, all using it to block child abuse images on their platforms. The technique works by converting an image to black-and-white and then breaking it into a grid. The resulting intensity gradients in each cell are used to create a photo’s DNA. PhotoDNA hashes are resistant to alterations, whether it’s resizing, resaving or digital editing, which makes them pretty powerful digital identifiers of harmful content.

The same companies that implemented PhotoDNA recently decided to use hash based technologies to curb the spread of online terrorist content. The outcome of this collaboration is a shared database of hashes of previously removed images and videos. Facebook, Microsoft, Twitter and YouTube are adding hashes of illegal files to the database for the benefit of others interested in identifying similar content on their services, in reviewing this content against respective policies or in removing matching content as appropriate.

The cost of this particular initiative is not known but, for example, we know that YouTube’s ContentID was anything but cheap – 60 million US dollars. Soundcloud’s system was cheaper but still quite expensive – 5 million euros. And although PhotoDNA is now available as a free cloud service, its use inevitably involves staff costs that not all SMEs may be able to afford. Indeed, costs are one of two reasons why automatic detection technologies are not widely adopted by (smaller) intermediaries; the second one is fear of losing liability protection. Article 14 of the E-Commerce Directive grants information society service providers – the term used in the legislation – immunity on condition that (a) they have no “actual knowledge of illegal activity or information” and (b) if they do obtain such knowledge they “act expeditiously to remove or disable access to the information.” But what counts as expeditious removal?

Takedown time

In Germany’s case, as mentioned before, it’s 24 hours if the content is easy to verify and seven days if the work requires more effort. The Commission is less prescriptive in its guidelines but considers the situation in which removals take more than a week “unsustainable.” In Austria, the Supreme Court also considered removals lasting one week as being too long. However, in Capitol Records v. Vimeo, the district court found that “given the number of infringing videos [170] at issue, the three and one-half week period it took Vimeo to comply with the notice constitutes expeditious removal.”  The district court also held that a “one-day response time” to remove between one to six videos referenced in a notice was expeditious. Other courts have similarly held that removal of infringing content within one to several days of receiving a notice constituted expeditious removal.


Starting 2014 more and more intermediaries began publishing regular updates on notices and their outcomes in special transparency reports. The focus of most reports is IP related infringements although some also include data on government requests for private information. Some reports are available in machine readable formats, others as pdf documents. Some are published twice a year, others only once. The level of detail also varies from intermediary to intermediary. Twitter, for instance, provides statistics on a wide range of outcomes, including the number of takedown notices received, the number of accounts and tweets affected, the number of counter notices filed, the percentage of materials eventually restored. A platform like WordPress, on the other hand, shares data only on the number of DMCA takedown notices received, the percentage of copyright notices where some or all content was removed and the number of counter-notices submitted.

Information on notices may be patchy but at least it’s available for a handful of platforms (all of which tend to be US based by the way). In comparison, information on the effectiveness of automatic detection technologies is shrouded in complete secrecy. How effective is PhotoDNA? How many pieces of content are affected by ContentID daily? Can users dispute a claim if their content was blocked by one of these systems? The answer is – we don’t know because intermediaries are guarding this information like a trade secret. Some even fail to admit they have automatic systems in place. To this camp belong Vimeo and Dropbox, both of which use digital fingerprinting to prevent copyright infringements but say nothing about these measures on the official channels; relevant information can only be found on third party sites.

In its guidelines, the Commission is only asking for transparency reports “detailing the number and types of notices received.” Considering that no such information has yet been published by European platforms, the first transparency report by Soundcloud, Dailymotion, Allegro, DaWanda or any other strictly EU-based intermediary would be a major milestone. But it’s important to also keep in mind the secrecy surrounding automatic detection systems; the Commission should be explicit about the issue in the future if it’s serious about promoting complete transparency.

Internet Intermediaries: Reality and Issues in Europe’s Liability Regime


Today 47% of the world population, or just over three billion people, are using the internet. The use of and interaction in the vast enabling environment that is the World Wide Web has been made possible by the work of online intermediaries, a diverse group of actors that facilitate interaction and transactions between third parties on the internet. Intermediary roles and types vary significantly: some provide access to the web either through wire-based or wireless communication technology; others offer hosting, payment, storage and on-demand services like music streaming; and then there are those who aid in navigation of online space (e.g. search engines) and those who create social hubs where people can come together to share content, news, advice and much more. Without intermediaries there would be no access to the internet, no opportunities to share information with others and no collaborative economy that changed the way we view ownership and property use.

In the early days of dot com era intermediaries were subject to limited regulation. But as the internet evolved, pressure on intermediaries to police their content grew concomitantly. Today intermediary liability – legal responsibility of intermediaries for illegal or harmful activities performed by users through their services – can occur in many circumstances and as a result of many issues, including those related to child sexual abuse material (CSAM), hate speech, racist and xenophobic content, defamation, infringement of intellectual property (IP) rights, terrorist and extremist content, illegal offers related to gambling, pharmaceutical products and banking, the sale of counterfeit goods, as well as malware, spam and data protection infringements.

Illegal Content

Just exactly how much of this content exists online at any one point is hard to quantify. For one, there is no single repository that aggregates data across the board. For another, different sources tend to specialise in different kinds of illegal content, making the creation of a complete picture of how much illegal online material was out there, say, a year ago rather difficult. For example, online intermediaries like Google, Twitter and Tumblr report extensively on copyrights and trademark infringements, but hardly anything on CSAM. By contrast, national hotlines prioritise CSAM and only some (e.g. hotline IE) report notices related to financial scam, violence and hate speech. Additional IP related information can be found in Lumen database and in the publications of some actors from the rightsholders camp, such as the Software Alliance, which publishes information on unlicensed software installation for several years and world regions, including Europe. But many other, equally important types (e.g. malware, spam) remain un- or under-reported.

Of course, not all content reported online is illegal. Some complaints draw attention to material that is actually legal under national laws, and some reports can be disputed thanks to counter-notification procedures provided by many online intermediaries. But even allowing for material which was wrongly identified as illegal or eventually restored, the sheer volume of illegal internet content is staggering. For example, in the year to 12 January 2017, Google removed 916 million URLs that affected 353,000 websites. Between January and June 2016, Twitter received 24,874 takedown notice requests, 75% of which led to the removal of content. Over the same period, Tumblr received 12,864 notices, which led to the removal of 97,403 pieces of content and 61,053 posts and the termination of 1,558 accounts. Considering this is just a small fraction of the actual extent of the problem, it is hardly surprising that illegal online content has very significant implications for the stakeholders affected, the economy and society at large.

Impact on Stakeholders

Take the most obvious victim – the music industry. Before Napster emerged on the file sharing scene in 1999, global music market was worth $37 billion. Although the service disappeared a couple of years later, the industry has suffered greatly from the drop in sales that file sharing platforms like Napster contributed to. (In 2015, record industry’s income amounted to just $15 billion.)

It goes without saying that online piracy and revenue loss go hand in hand. In 2014, EU observatory on IP infringements reported that European music industry lost 5.2%, or €170 million, of its total annual sales due to physical and digital piracy. Studies for other countries report similarly negative effects. For example, in his study Stephen Siwek estimated a total sales loss for the U.S. economy of $58 billion per year, along with 373,000 jobs lost as a result of global piracy. All of that translated further into an earnings loss of $16.3 billion and reduced tax collections of $2.6 billion.

The trade-off between the benefits of innovation and the consequences of its disruptive potential makes regulating digital space no easy task for a policymaker. On one side there are those who call for a more restrictive approach, arguing that laxed environment is conducive to unsustainable loss of revenue, reduced social welfare and wider economic consequences such as unemployment. On the other are those who say that internet access is a human right and that people should have access to a virtual public space just as they have the right to assemble offline. Digital rights activists further add that enforcing restrictive requirements on online content amounts to restraining people’s access to a potentially unlimited market of opportunities and many different kinds of experiences, including social and cultural. Accessing and making online content available to all can therefore be intrinsically linked to individuals’ right to freedom of expression.

Liability Regime in Europe

In the EU, intermediary liability regime is governed by the E-Commerce Directive which shields from liability intermediaries that act as mere conduit, caching and hosting providers, on condition that “the provider does not have actual knowledge of illegal activity or information and… the provider, upon obtaining such knowledge or awareness, acts expeditiously to remove or to disable access to the information” (Article 14).

Although the directive prevents member states from imposing general monitoring obligation on service providers, it states that “member states may establish obligations for information society service providers promptly to inform the competent public authorities of alleged illegal activities undertaken or information provided by recipients of their service or obligations to communicate to the competent authorities, at their request” (Article 15). This means that there is no absolute protection in the directive for internet intermediaries and they are not immune from prosecution and liability. Such legal uncertainty, combined with the lack of clarity on the requirements for notice-and-action procedures, is a cause of considerable anxiety for many intermediaries.


The realities on the ground, coupled with findings from several consultation rounds carried out by the European Commission, point to several issues in the current liability regime that must be resolved as soon as possible, starting with the role of intermediaries in the information transmission process.

  • Intermediary role. There is a need to clarify the meaning behind “mere technical, automatic and passive nature” of information transmission that intermediaries help facilitate. Some stakeholder groups, including the intermediaries themselves, find the meaning sufficiently clear. However, this is not the case with notice providers who think that owing to the lack of clarity the concept cannot be applied uniformly.
  • Intermediary type. Internet changes at breakneck speed. This means, among many other things, that definitions that are valid today can become obsolete very quickly. As digital space rapidly expands with new services and platforms, it is worth asking whether the definition of an online intermediary is still apt. More than half of the respondents to the consultation (notice providers, researchers, right holders and civil rights organisations) believe that there is a need to establish new categories or to clarify existing ones.
  • Notice-and-action. There is also a lack of clarity over how a typical N&A procedure may/should be implemented and very little information about the actual practices/processes is shared by some types of intermediaries, notably ISPs. Further, consultation findings show that there is a preference to have different N&A approaches towards different types of illegal content.
  • Counter-notification. Counter-notification is the norm in many complaint procedures administered by tech companies like Google, Twitter, Tumblr and VIMEO, to name just a few. But should this norm be also embraced by EU-based intermediaries, and if so – which ones? (As to the first question, the answer appears to be a firm yes based on consultation results, as over 80% of respondents agreed that this mechanism should be introduced).
  • Illegal content and a duty of care. Illegal content is recognised as a serious problem and many intermediaries have taken proactive – sometimes very advanced – voluntary measures to remove it from their sites. But despite this the question about the duty of care and the extent to which it should be applied continues to dominate discussions on intermediary liability. Based on consultation results, over half of the respondents reject the duty of care arguing that it will be too costly, raise entry barriers and lead to abuses. Many intermediaries prefer that duty of care remains voluntary. However, holders of IP rights believe that actions such as filtering, blocking and internal policies should be part of an obligation.

Sharing International Best Practice in Oman

Julia had the honour of delivering the keynote address at the Fifth Edition of the Sultan Qaboos Award for Excellence in eGovernment in Muscat, Oman, on 11-12 January 2016. Speaking before a packed audience of government entities, civil society representatives and private sector stakeholders, Julia used the address to stress the growing importance which the UN eGovernment Survey is placing upon open, collaborative and citizen centric government.  ‘To achieve the ultimate target of making citizens happy,’ Julia argued, ‘government must constantly keep pace with the rapid pace of digital transformation. Mobile, Open Data and Cloud have transformed our interactions with each other and the business community. World leading public administrations know this and aspire to offer services that are as good as, if not better than, those offered by internet giants like Amazon, Apple and Google.’ Following her address, ITA, Oman’s central eGovernment agency, invited Julia to host a special workshop with their national portal team to review international best practice in key areas such as citizen engagement and Open Data.  Julia concluded her visit with a closing speech on the innovative use of social media to engage citizens and improve service delivery. A brief post-event interview with Julia is available here.

This slideshow requires JavaScript.


The Tip of the Spear: Connecting Big Data Project Management with Enterprise Data Strategy

By Dennis D. McDonald. Prepared for the ATARC Federal Big Data Summit, Dec. 8, 2015, Marriott Metro Center, Washington, DC .

berries3“If data analysis is Big Data’s “tip of the spear” when it comes to delivering data-dependent value to customers or clients, we also must address how that spear is shaped, sharpened, aimed, and thrown – and, of course, whether or not it hits its intended target. We also want the processes associated with throwing that spear to be both effective and efficient.”

In Meeting the Mission of Transportation Safety , Richard McKinney, U.S. Department of Transportation’s CIO, describes four components for what can be called an “enterprise data strategy”:

  • Data governance
  • Data sharing
  • Data standards
  • Data analysis

He also mentions additional factors relevant to DOT’s data strategy:

  1. The volume of data is increasing and we need to be ready for it.
  2. Managing data is not the same as analyzing it.
  3. We need to be thinking now about what type of analysis we need to be doing and what resources will be needed to do the analysis.

Based on the 20+ personal, telephone, and email interviews I’ve conducted so far  as part of my big data project management  research I would add a fourth item to McKinney’s list:

4. We need to devote at least as much time to planning and managing the people and business processes that make data analysis possible as we devote to the analysis process itself and the technologies that support it.

This Paper’s Target Audience

This paper is for managers and executives who are looking for ways to add “big data” benefits to their organization’s operations.

Perhaps the organization is experiencing a significant increase in the type and volume of data it is generating and wants to make sure it points its evolving analytical capabilities at addressing its most important goals, objectives, or problems. Or, perhaps the organization just wants to do a better job of making use of the data it already has.

Tip of the Spear

If data analysis is Big Data’s “tip of the spear” when it comes to delivering data-driven value to customers or clients, we also must address how that spear is shaped, sharpened, aimed, and thrown – and, of course, whether or not it hits its intended target.

Management needs to involve both business and technical resources in its planning process. While a variety of technical skills and capabilities may be required to develop and govern an effective data analytics program, the focus of this and the other reports in this series is not on technology or analysis tools per se but on how to make sure that data analysis and data governance processes are driven by business requirements and the delivery of practical benefits to the organization.


Making the data analysis process – the tip of the Big Data spear — effective and efficient is where good project planning and management come in. Challenges to doing this in connection with data intensive projects are many and include:

  • Siloes. Data are often generated and managed in system- or mission-specific siloes. As a result, creating and implementing an effective enterprise-level data strategy that rises above and encompasses multiple programs, systems, and/or missions requires not just data analysis skills but a mix of technical, organizational, and political skills – not just good “project management.”
  • Sharing. Making data accessible and useful often means that data need to be shared with systems and processes outside the control of those who “own” the data to be analyzed. Key steps in sharing data are that (a) data need to be identified and inventoried, and (b) technical and business ownership of the inventories data must be determined. In many organizations this inventorying is easier said than done and may require both manual and automated approaches to creating the necessary inventories.
  • Standards. Efficient and sustainable analysis of data and metadata may require development or implementation of data standards. Existence and use of such standards differs by industry, data type, and system. The costs for developing and adopting standards to facilitate data sharing and analysis will also vary and may have cost and schedule implications at the project, program, enterprise, and industry or community levels.
  • Delivering value. Modern data analysis tools and techniques provide mechanisms to identify patterns and trends from the increasing volumes of data generated by a steadily widening variety of data capture mechanisms. Challenges in predicting what will be found when data are analyzed places a premium on making sure we are asking the right questions. This in turn impacts our ability to justify project expenditures in advance.

Portfolio Management

Responding to the above challenges requires not only project management skills but also a project planning process that takes into consideration the organization’s goals and objectives.

As one of my interviewees suggested, the challenge faced in complex “big data” projects has just as much if not more to do with overall strategy and “portfolio management” as with how individual projects are planned and managed. Effectively designing and governing a portfolio of projects and processes requires not only an understanding of how the portfolio supports (relates to, is aligned with, interacts with) the organization’s objectives. It should also incorporate a rational process for defining project requirements and then governing how the organization’s resources are managed and applied over time.

Given how pervasive and fundamental data are to an organization’s operation, skill in data science and analytics is a necessary element but will not be in many cases be a sufficient contributor to success. Technical and analytical skills must be accompanied by effective planning, oversight, and management in order to ensure that the data analysis “spear” is being thrown in the right direction. While it is not unusual to assign responsibility for big data projects to the IT department, both business and functional leaders from outside will also need to be involved.

Delivering Value Quickly

Ideally a defined portfolio of projects will support an organization’s strategic plan and the goals or missions the organization is charged with pursuing. In the real world, though, we can’t spend all our time planning, we may also need to “get tactical” by delivering value to the customer or client as quickly as possible.

In organizations that are not historically “data centric” or where management and staff have a low level of data literacy, an early demonstration of value from a targeted data analysis initiative will be important.

Balancing Tactics and Strategy

Unfortunately, challenges such as those identified above in many cases cannot always be addressed effectively in tactically focused short term projects. For example, convincing “data silo” owners to cooperate may take time given how the organization has been traditionally structured and managed.

Attention to enterprise-level data strategy while delivering useful results in the short term has implications beyond what is being attempted in an individual project’s scope. Treating data as an enterprise resource may even require changes to how the enterprise itself is managed.

As we all know, it’s not unusual for change to be resisted!

It’s not unusual for a tactically-focused project that’s delivering a practical data-based deliverable to uncover the need for a more global (or strategic) approach to managing data, metadata, data security, privacy, or data quality. In such instances it makes sense for the project manager, when communicating with project stakeholders to clearly identify strategic concerns along with reporting on current work. Experienced project managers will already be doing this.

An effective enterprise level data strategy will be one that balances the management of a portfolio of individual data intensive “agile” projects with parallel development of an upgraded enterprise data strategy and governance process. Doing one without the other could have negative consequences, for example:

  • Focusing only on a narrowly defined data intensive analytics project by itself may generate immediate value through frequent useful deliverables but may not address underlying technical process issues that impact long-term efficiency and sustainability.
  • Focusing only on an enterprise data strategy without delivering tactical benefits reduces the possibility that that less data-savvy managers understand the “big picture” down the road.

As experienced project managers know, concentrating on “quick and dirty” or “low hanging fruit” when under the gun to deliver value to a client in the short term can generate short term benefits.

They also know this approach can actually increase costs over time if strategic data management issues related to data standards or quality are repeatedly kicked “down the road.”

Also, delivering a “strategy” without also engaging users in development of real-world analytical deliverables might also mean that strategically important recommendations ends up gathering dust on the shelf somewhere.

Communication Strategy

As experienced project managers understand, one of the most important ingredients in successful project management is communication among project staff, communication with the client, and communication with stakeholders. Even when focusing on delivering incremental value quickly, we need communications about project activities, especially among key stakeholders, to focus both on tactical as well as strategic objectives. This may require accommodating a variety of communication styles as well as different levels of data and analytical literacy, especially when both business-focused and technology- or analytics-focused staff are involved.

The project manager must think carefully about what it will take not just to deliver a useful result now but also what it will take to make sure that participants and stakeholders understand the meaning and significance of what is being delivered. This may be straightforward in organizations that are already engaged in heavily data-centric activities. In other types of organizations more explanation and learning will be required.

Where to Start

A planning process that takes into account how the unique needs of the organization interact with an improved data management strategy is what is needed. Working without a plan that links improved data management with both tactical and strategic benefits is a recipe for, at best, wasted time or money. At worst, working without such a plan can lead to financial or organizational disaster.

In the case of big data, a revolution is occurring both in the management of steadily increasing volumes of data and in how data are organized, stored, and analyzed. Even managers of very traditional data collection and publishing operations are seeking ways to improve how they manage and use data based on what they hear about the potential for “data science” and “data engineering.”

What I’ve been finding, though, is not that project management practices need to change. Instead, what organizations want and need to accomplish through improved data access and management needs to be better defined — and ideally, quantified — so that investments in time, technology, and other data-related resources are planned and made wisely.

This means that the “front end” of the project and program planning process needs to address fundamentals such as business requirements, business strategy, use cases, return on investment, and the related issues of security, privacy, standardization, competition, and innovation.

Whatever specifics are being addressed in the short term, the strategic implications of improved data management need to be addressed as well. We may find, for example, that what is needed initially is not a move to an entirely new data management architecture but the creation of business processes that make better use of the data resources that currently exist.

What’s Next

Among other things I’ll be turning my attention next posts in this series to address this planning process more specifically, possibly beginning with the relationship between communication and requirements definition.

If you or a client think you need help in this area, please let me know; my contact information is below. I will be happy to explore with you confidentially how I can help you to rapidly put these ideas to work in your own planning process.


Thanks are due the following for sharing their thoughts with me about big data project management: Aldo Bello, Kirk Borne, Clive Boulton, Doug Brockway, Ana Ferreras, Keith Gates, Douglas Glenn, Jennifer Goodwin, Jason Hare, Christina Ho, Randy Howard, Catherine Ives, Ian Kalin, Michael Kaplan, Jim Lola, David McClure, Jim McLennan, Trevor Monroe, Brian Pagels, John Parkinson, Dan Ruggles, Nelson Searles, Sankar Subramanian, and Tom Suder.

About the Author:

Dennis D. McDonald, Ph.D. is an independent Washington DC area management consultant. Services include project and program management, project plan development, requirements analysis, strategic planning, preproposal research and analysis, proposal development and costing, and marketing and communication support. Reach him by phone at 703-402-7382 or by email at ddmcd@outlook.com. His website is here: www.ddmcd.com.


Talking Happy Cities @ Slovakia

julian ITAPA

By Julian Bowrey 

When the President of the Republic calls the government web portal a “sad memorial of the internet age” I knew the 14th ITAPA conference in Bratislava last week wasn’t going to be a triumphant procession of speakers acclaiming the success of their digital government projects. In fact, throughout the event there was a refreshing willingness on the part of Slovak colleagues to be self-critical. And this openness made the undoubted successes even more laudable. Among many excellent digital government projects showcased, I was particularly impressed by the Ministry of Justice projects on implementing an Electronic Identification Card and electronic tagging of offenders as well as the digitisation of the Museum of the Slovak National Uprising.

Overall the conference themes very much echoed the concerns of digital government professionals worldwide: utilising the cloud, online identify management, cyber-security, marketing and take up of digital government services including the role of the digital community to promote and act as advocates for these new services.

It was also fascinating to see the impact of Estonia as a comparator and driver of change for Slovak colleagues; notwithstanding the Baltic republic having a 15 year start on Slovakia and one quarter of the population. This healthy competition was clearly a spur to local digital government projects.

Several speakers addressed wider issues of building digital capacity within society. Some of this obviously related to economic modernisation and improving productivity, but also recognising the importance of improving digital skills to build a strong civil society. Dion Rudnicki of IBM spoke about the need for “U shaped” government; top-down from government to citizens then per to per exchange followed by bottom-up message back to government. A message that complemented my talk about the need to make our “smart cities” citizen friendly and fun as well as efficient. And to be investing as much in “smart citizens” as in “smart cities”.

The final plenary session introduced by the Minister of the Economy concluded the conference on a positive note. Looking forward to the next programme phase running until 2020, participants prioritised promoting use of the new digital services, capturing and managing existing data better and making better use of data analytics to personalise new services. Two other emerging themes were building services for smart phone and mobile, and, crucially, keeping existing teams together to build momentum for change. It was a really interesting and enjoyable event with some really powerful insights.

So why was the President so critical? Is he a fan of GOV.UK and wants to chivvy his Ministers and officials? Probably, but more tactically there is a general election in March and perhaps the President was also surreptitiously limbering up for that campaign?

Problems and Opportunities with Big Data: a Project Management Perspective

By Dennis D. McDonald[1]

Being able to make wise investments in “big data” capabilities may require more collaborative approaches to project management and decision-making than are currently being practiced. Two recent articles illustrate this:

The first article states an often heard complaint that, despite much marketing hype, real-world examples of big data value are few:

“It does not help when so-called experts suggest that an information revolution is changing–and will continue to change–the nature of the workplace itself. Big data projects can have a transformative effect on business operations and processes. Yet evidence of a knowledge lead revolt in the offices of global blue chips is thin on the ground. Instead large organizations seem to be drowning under a sea of information.”

The second article focuses on the opportunity side of the equation. It addresses how technology management needs to evolve in the face of growing technical data handling capabilities:

“From the top of the stack, more users want access to more data and more combinations. And from the bottom of the stack, more data is available than ever before–some aggregated, much of it not. The only way for data professionals to deal with pressure of heterogeneity from both the top and bottom of the stack is to embrace a new approach to managing data that blends operations and collaboration to organize and deliver data from many sources to many users reliably with the provenance required to support reproducible data flows.”

My own interest — making organizational data more open and accessible to both internal and external user groups — dovetails with both perspectives.

Both views show how people are grappling with the challenge of using modern tools to make sense of the increasing volumes of data in digital form. If there is no clear understanding of what benefits will flow from big data investments, investment justification will be a challenge. Anyone who has ever sold a new tool or technique understands this. This is why practical demonstrations and understandable testimonials are so important.

It is clear that tools and technology are improving in ways that can make sophisticated data usage more accessible and immediate to those with the necessary knowledge. For example, the turnaround time for processing large amounts of data is dropping as new platforms and tools are introduced. Processing speed and the ability to combine analyses of different data types can be documented objectively and technically.

One question is, can we demonstrate the real world benefits of such techniques in meaningful ways without requiring management to possess data science degrees? Just talking about the benefits of speed and volume in technical terms is not enough. Showing how these benefits translate into useful insights and support for useful planning and decisionmaking is required.

Healthy skepticism

The first quotation above reflects a healthy skepticism brought on by much of the hype surrounding big data. Such skepticism is a good thing if it forces proponents to articulate the “whys” and “wherefores” instead of just the “hows” of big data. This need for proof is why supportive “case studies” are in such demand at meetings and conferences. Consultants and vendors understand this and is why so many stories about big data “successes” are industry sponsored.

The second article’s emphasis on infrastructure is typical of tech-oriented solutions that focus more on the technology side of the equation. Yet there is more here than just a veneer of newness. Combining “big data” with “DevOps” makes great sense. I especially like the emphasis on the need for communication and collaboration which are critical to figuring out how best to grapple with taking advantage of powerful new tools.              

Déjà vu

I have a sense of déjà vu when reading articles like these. “Tech hype” has always been with us. Mature tech managers and their bosses long ago mastered the art of healthy skepticism when hearing vendors expound on new technologies. A “show me” attitude is a healthy one to promote especially when price tags are high, a lot of changes are required, and business value may be difficult to pin down.

There’s always a lot of faith and trust required at this stage of technology adoption. That tech vendors are investing heavily in tools, advertising, marketing, and hype need not divert us from asking the tough questions. Inevitably discussions about how best to take advantage of new tools will lead to serious discussions about strategy, governance, quality, and costs.


Pushback from the organization can come in many forms ranging from good old-fashioned resistance to change (how many times have we heard, “You’ve got to change the culture!”) to major implosions of high-profile projects or vendors. What may be different this time around with big data may be the speed with which new tools are introduced and the seeming ease with which tools can be used to analyze and visualize vast amounts of data quickly.

Range of changes

New data tools and technologies may also stimulate the need for changes to current management processes which, more often than not, generates pushback from those being changed. Adopting big data tools and process changes may be associated with a range of organizational changes including:

  • The move to cloud services as a replacement for current infrastructure.
  • The need to learn new tools and techniques.
  • Resistance from business process owners who don’t want to change.
  • Overemphasis on tools and technologies while shortchanging business and strategy.
  • The need to align data services with core business needs, not just with “easy to do” and “low hanging fruit” initiatives.

While it’s important for IT staff to understand new data tools, employees and managers in all functional areas are impacted and must be able to articulate what they want from the data and from the new tools.

This is where the need for collaboration and communication emerge along with the need for basic project management techniques and support. Stakeholders need to be identified and brought on board. Resources need to allocated and managed. Progress needs to be tracked and reported in some fashion. New products and services need to be aligned with goals and objectives that are important to the organization. Inevitable changes need to be addressed as learning takes place. Most important, project goals and objectives need to be clear and understandable to all.

No best practices

I think that, regarding the first article cited above, skepticism is healthy but needs to be specific to the organization’s needs, not general. Searching for “best practices” and examples of whether or not other organizations are able to take advantage of big data applications may have little relevance to one’s own organization. Understanding one’s own needs and requirements must be the starting place, otherwise we run the risk of doing the “same old things” only with new tools.

One criticism of the second article is related to this. That is, while I do believe that we may need to re-think how we manage infrastructure and IT services to take advantage of new tools, services, and platforms, we also need to pay attention to how priorities are set and initiatives managed. Again, that’s basic project management and points out the need for communication and collaboration in how priorities are set and how solutions are implemented.

Whether we call this “DevOps” or “DataOps” is not the issue. The issue is whether or not we can effectively manage projects and programs that take advantage of big data while involving all necessary stakeholders throughout every point in the data value chain, not just IT staff and data analysts.

Data literacy and strategic alignment

If it is inevitable in today’s fast paced world that many fingers are going to be stuck in the “data management pie,” we need to make sure that the heads operating those fingers have a basic understanding of how data are generated, managed, and used; let’s call this “data literacy.” Without basic data literacy participants risk talking past each other with the end result being inefficiency, blind alleys, and disappointment.

The connection between better data services needs to be aligned clearly with the organization’s goals and objectives. Again, this is a basic requirement for effective project management. Management, IT, and data analytics experts need to communicate and collaborate right from the start so that everyone is on the same page.

Related reading:


The author is grateful to Julia Glidden of 21c Consultancy Ltd for her helpful comments on an earlier draft of this article.

[1] Copyright © 2015 by Dennis D. McDonald, Ph.D. Dennis is a management consultant based in Alexandria, Virginia. His experience includes consulting company ownership and management, database publishing and data transformation, managing the integration of large systems, corporate technology strategy, social media adoption, statistical research, open data, and IT cost analysis. Clients have included the U.S. Department of Veterans Affairs, the U.S. Environmental Protection Agency, the National Academy of Engineering, and the National Library of Medicine. He has worked as a project manager, analyst, and researcher for both public and private sector clients throughout the U.S. and in Europe, Egypt, and China. His web site is located at www.ddmcd.com and his email address is ddmcd@yahoo.com. On Twitter he is @ddmcd.

6 Top Tips for Building a Smart City with Data

By Dr Julia Glidden and Susie Ruston McAleer

How do you make open data an achievable reality for every city, regardless of its size or budget?


Technology hub Issy-les-Moulineaux, Paris, opened over 75 datasets and created new apps, following open data guidelines and interactive workshops. CC BY 2.0, uploaded bycouscouschocolat.

That’s the question Citadel on the Move set out to answer five years ago. The £4M project, funded by the European Commission, has since helped over 140 local government organisations across six continents open and use data.

While initial visions of the smart city centred around a tech-driven, top-down approach that placed government firmly in the driving seat, in recent years a more collaborative, bottom-up vision has begun to emerge. Within this more ‘human-centric’ conception, a truly ‘smart city’ is seen as one that uses the full range of new technologies like social media, mobile phones and open APIs.

A smart city will use these technologies to gather knowledge and data from urban settings, open up the data in order to solve problems, co-create value with citizens and become ‘smart’ in the process. Under this new paradigm, citizens and governments are seen as equal participants in the creation of innovative new data-driven solutions to urban challenges.

From navigating the minefield of open data obstacles to making a local area a magnet for data-driven businesses, our work on Citadel has highlighted a number of key ingredients for smart city success.

Strong leadership

First and foremost, smart cities are supported by clear leadership and a desire or need to innovate. The policies and key IT infrastructures might not be in place, but if a city’s leadership has the vision and drive to help unlock the power of open data and innovation, the rest of the pieces are likely to fall in order.

Issy-les-Moulineaux, a small city on the outskirts of Paris with a strong reputation for technological innovation, recognised the value of open data but lacked the internal capacity and expertise to kick-start an open data culture. By following practical guidelines and interactive workshops, Issy was able to open over 75 datasets and create new apps in areas ranging from trees to tourism. Building on this work, Issy has recently partnered with OpenDataSoft, one of France’s top internet startups, to install a stand-alone open data platform, and release a novel new financial transparency app) which can be used by other local government entities around the world.

Based on our experiences with Athens and Issy, alongside all our other partners and associates, we’ve drawn up a simple set of business recommendations for Local Authorities everywhere to open their data and build a smart city.

  1. Understand your business casefor opening data, whether it’s to boost transparency, generate cost savings or unlock innovation. This knowledge will help you form your own objectives.
  2. Once your business case is in place you shouldstart opening data as soon as you can, to avoid your city being left behind the technology and policy curve.
  3. Ensure your data is broadlycompliant with industry standards and will be re-usable. Citadel offers detailed recommendations for local government on open data, formats, structuring and licensing.
  4. To ensure you unlock the value hidden within your data,create a local data ecosystem of tools and guidelines that will help your community use your data. You can find resources for both novice and experienced developershere.
  5. Understand your communityand their needs to ensure you engage them and incentivise participation accordingly – from simple public thank yous and recognition via social media to cash prizes for developing services to solve specific challenges.
  6. Be prepared to come up against usability challenges, and help your community to mitigate common problems.

This can be done in various ways:

  1. a) Use ourconvertorto easily create data files that developers can access and use to create new services.
  2. b) Avoid character coding challenges in non-English datasets by saving Excel or CSV files as UTF-8.
  3. c) Use freely availableGeo batch codersto add latitude and longitude coordinates to address data so it can be represented in map form, using our Application Generator or other visualisation tools.
  4. d) Use recognised open licences, such asCreative Commons 4.0, so your community is in no doubt that it can use your data for innovation without restriction.
  5. e) Add new datasets to your ecosystem by gaining buy-in throughout your authority. Use tools like the ‘Apps4Dummies’ workshop to quickly demonstrate open data value to leadership.
  6. f) Share any new developments with your community and the wider Citadel community.

Over 5 years, Citadel has seen first-hand how uneven ‘starting’ positions with open data are:

  • 17% of Citadel’s 140 Associates had no previous contact with open data
  • 24% had little experience of open data (some data but no city portal or systematic release)
  • 47% had some experience of open data (a city portal or systematic release but no clear policy on open data publication and updates)
  • 12% had advanced experience of open data (a portal or systematic release and a policy of open data publication and updates)

Regardless of level of experience, however, the Citadel team has found one consistent theme: with the right leadership and support in place any city or town can become an open data advocate.

Dr Julia Glidden is President of 21c Consultancy and Senior Research Fellow at Vrije Universiteit, Brussels. Susie Ruston McAleer is Managing Partner at 21c Consultancy http://www.21cconsultancy.com. Follow @21cDatahttps://twitter.com/21cDat*a on Twitter

The Smart City as a Sharing City?

Dr. Julia Glidden, President, 21c Consultancy, Senior Research Fellow, Vrije Universiteit Brussels

Whilst initial visions of the Smart City encompassed a tech-driven, top-down vision that placed government firmly in the driving seat, in recent years a more collaborative, bottom up vision has begun to emerge.   Within this more ‘human-centric’ conception, a truly Smart City is seen as one that uses the full range of smart technologies e.g. social media, mobile, open APIs to facilitate sharing and co-creation – ultimately making cities more liveable, humane and sustainable in the process.

Under this new paradigm, citizens and government are equal participants in the creation of innovative new solutions to urban challenges. Tech-enabled sharing is the mechanism. Data is the fuel. Empowered by this new paradigm, citizens around the world are actually in the process of rebranding the Smart City as the Sharing City:  from pets and gardens through to bedrooms and cars, a rapidly increasing number are combining smart technologies and data to embrace the notion of ‘collaborative consumption’ or the borrowing and sharing of goods and services.[1]

Toward a New Form of Public-Private Sector Partnership

Google is now by far the most powerful gateway to the Internet’s vast commercial potential. AirBnB, despite owning no hotels, rents out more rooms than the Hilton Hotel Chain has in its entire portfolio.  Uber, despite owning no cars of its own, is now valued at more than many of the world’s largest companies and Fund IT has one of the highest success rates for reward-based crowd-funding sites anywhere in the world.  Yet, to be sustainable a truly Smart, City cannot rely on the Private Sector alone.  Instead, having collected and sat on vast quantities of data for many years, the Public Sector must contribute to the co-creation mix by making this critical raw material open and freely accessible – in a machine-readable format – to citizens and businesses alike.

From fueling the creation of new public services to forging new markets, businesses and jobs, Open Data has the potential to empower citizens, entrepreneurs and developers to spark new economic development, improve service delivery and ultimately change how government works.[2]  Despite all the buzz about the promise of Open Data to create a smarter, more sustainable urban model, however, one still hears very little about the practical steps that a public administrator needs to take to open government data and unleash Public &Private Sector Innovation.  Too often Open Data advocates, for all their commitment and passion, forget that for many cities and towns, especially smaller ones with limited resources, opening and using data is easier said than done.[3]

What it Takes to Get There?

From navigating the minefield of open data obstacles to making a local area a magnet for data-driven businesses, our work on MyNeighbourhood, a €4.8M EU-funded project, has highlighted a number of key ingredients for success. First and foremost, truly Smart Cities are supported by clear leadership from above and a desire and/or need to innovate.  The policies and key IT infrastructures might not be in place.  However, if a city’s leadership has the vision and drive to help unlock the power of Open Data & Innovation, we have found that the rest of the pieces are likely to fall in order.

There will always be certain circumstances in which it is simply not feasible to freely open all data. Wherever possible, though, MyNeighbourhood, in line with its sister project Citadel,[4] strongly recommends adherence to the norms of the Open Data Commons wherein the users and providers of data agree to share and share alike.[5] In addition, MyNeighbourhood recommends that cities take the following steps when opening their data:

  1. Select the Data Set: Remember – simple and small can be good! The key thing is that the data is publically available and has no IPR restrictions on the way it can be used, reused or redistributed.
  2. Publish the Data Set: For ease of use, try to use non-proprietary formats (such as CSV instead of Excel) and structured formats (e.g. not scanned images, PDF etc). But most important of all, put your data out there. If someone wants to use it they will find a way!
  3. Publicise the Data Set: Data needs to be discovered and discoverable to be used. Make sure to post your data on your website or a central catalogue, and tell your citizens it is there!
  1. Engage your Community: From simple public thank you’s and recognition via social media to cash prizes for developing services to solve specific challenges, enlist local stakeholders to help create a innovation ecosystem for openness and sharing.
  1. Keep your Data Up-to-Date: Expectations for current, if not real-time, data are growing. Remember to publish new files as amended or updated data becomes available.

Where We Are Now?                                                                   

Open Data leaders like the United Kingdom and United States now explicitly advocate building and maintaining useful Application Programming Interfaces (APIs) – which enable the public and private sectors alike to dynamically access and share government data – to drive innovation.[6]   API’s represent an important emerging Open Data trend (and pathway to open innovation & co-creation) in that they explicitly acknowledge the value of publishing open government data for third party use.[7]

However, as a recent report by McKinsey notes, Open Data (and the sharing it facilitates) still faces barriers, including privacy concerns and the need for legal and regulatory frameworks. [8]  From health and safety through to taxation, finance and the fate of established industries, questions abound as to how best to respond to the shared economy and the potential for sustainable innovation, jobs and economic growth it unleashes.  To address these challenges in a manner that keeps pace with evolving citizen expectations, many governments are beginning to broaden their digital agenda beyond the efficient and effective delivery of existing public services to toward a more holistic emphasis on what it actually takes to positively channel the disruptive realities underpinning the notion of the Smart City as a Sharing City.

What Have We Learned?

Sharing facilitates public sector savings, environmental sustainability, community empowerment, social cohesion & new business models.  Equally important, Consumers and Business now expect the choice, convenience and cost effectiveness that ‘sharing’ enables.  Consequently, governments everywhere now have little choice but to either adapt to the disruptive change inherent in the Sharing Economy or be left behind.

Regardless of level of experience and preparedness, however, the MyNeighbourhood team has found one consistent theme behind the Smart City as a Sharing City:  with the right leadership and support in place any city or town can start to become an innovation platform for sharing.  It is not necessary nor even feasible to have all the answers – no one does in this era of fast paced change.  But it is necessary to have vision and the willingness to embrace openness and change rather than wish it a way.

[1] A bewildering array of collaborative business endeavours has wreaked havoc on a myriad of industries, transforming the traditional landscape of business and society (Botsman & Rogers, 2010; Gansky, 2010)

[2] McKinsey Report notes seven sectors alone could generate more than $3 trillion a year in additional value as a result of open data www.mckinsey.com/insights/business_technology/open_data_unlocking_innovation_and_performance_with_liquid_information

[3] The EUROCITIES Guidebook is a notable exception to this rule.  The guidebook is written by and for cities, and offers tailored information and practical guidelines for cities with limited resources and/or knowledge.  See: http://www.eurocities.eu/eurocities/publications/EUROCITIES-open-data-guidebook-open-data-working-group-publication-WSPO-9CTHZE

[4] http://www.citadelonthemove.eu Citadel on the Move is a £4M European Commission-funded Smart City flagship that has helped over 140 local government organisations across all six continents to open and use data to unleash innovation.

[5] http://opendatacommons.org/norms/odc-by-sa/

[6] APIS are how modern Internet software and apps talk to each other using low cost, existing infrastructure that already exists.   https://medium.com/@ShorensteinCtr/wholesale-government-open-data-and-apis-7d5502f9e2be

[7] http://techpresident.com/news/24829/free-data-debate-over-apis-and-open-government

[8] www.mckinsey.com/insights/business_technology/open_data_unlocking_innovation_and_performance_with_liquid_information

Open data: gold mine or gold bust?

“On 12 December 2011, European Commission Vice President Neelie Kroes launched a European Open Data Strategy, boldly claiming that the European Commission would lead member states by example in a pioneering effort to ‘turn government data into gold.’ Kroes’ announcement – along with her argument that opening data could provide a €40-billion boost to the EU’s economy each year – was justifiably welcomed by open data advocates as the clearest example yet that the Open Data movement is here to stay.

Kroes’ impressive EU initiative is the latest in the seemingly inexorable advance of the move to make government data more transparent and accessible to us – the taxpayers – who, as Kroes rightly argues, ‘have already paid for this information.’ From the Obama administration’s 2008 Open Government Initiative – which promised to herald greater ‘transparency, participation and collaboration’ – to arguably even more extensive efforts by EU member states such as the United Kingdom and France, the public embrace of open data is undeniable.

Yet for all the outward commitment and hype, scratch beneath the surface of open data advocacy and one still finds a lurking band of ‘resistors’ armed with a seemingly unending array of reasons why ‘their organisation is special,’ ‘their data can’t possibly be open,’ ‘their constituents have special needs’. Take, for example, the leader of one national rail service who – incredible as it may sound – once openly argued that transport timetables could not possibly be made open because ‘rail travellers might miss their trains!’

If such statements are still being made at the national level, one shudders to think about the behind the scenes – let alone public – discussions that are currently taking place at the local level – where open data is only just now coming onto the radar. For sure, across Europe one can find ‘smart’ cities like Ghent pro-actively embracing open data as a natural means to kick-start local innovation and growth (see for example the use of geospatial data for the Ghent Festival). For every Ghent, however, one can find an infinitely greater number of smaller local authorities that are struggling not just to keep abreast of recent national and EU directives but also to understand where and indeed how to even begin to open data in a legally compliant and accessible manner.

This disconnect between the EU and national level and the local one presents a particular challenge for those like Neelie Kroes who hope that the open data movement will indeed ‘turn government data into gold’ – especially if by gold we mean more cost-efficient and effective public services as well as new wealth generation. The American politician Tip O’Neill once famously quipped that ‘all politics is local.’  Nowhere is this arguably truer than in the opportunity that open data presents to unleash the innovative potential of European citizens to collaborate with government in the co-creation and co-design of the types of modern, 21st-century public services that people want and need.

For all the talk of boarderless worlds and shrinking distances, most of us still live our lives locally, care about what happens most in our own backyard and are more likely to know better than anyone else about what our own neighbourhoods and communities need. Unfortunately, the enormous opportunity that open data – combined with ever more accessible and easy to use new social media tools – has to enable us to help drive this change will be five, perhaps even 10 years in the making (hardly any time at all for our army of ‘open data resistors’ but a virtual lifetime for the rest of us) unless the same level of energy, thought and commitment that is going in to open data at the EU and national level is applied locally.

Fortunately, alongside the growing number of supra-national EU Open Data initiatives (see for example:www.engage-project.eu), we now finally have a pan-European project devoted specifically to driving local level innovation and change. Launched on 1 February 2012, Citadel on the Move is co-funded by the European Commission Information Communication Technologies Policy Support Programme (ICT-PSP) for Smart Cities. The project is designed to respond to the unprecedented chance that the use of open data and social media presents to unleash the heretofore untapped potential of citizens to build the type of smart, ‘city-level’ services they want and need.

Beyond merely opening data for local-level use, Citadel on the Move aims to combine Open Data and Mobile Application tools to create new public services that can be used across Europe. Ultimately, Citadel on the Move seeks to advance nothing less than digital materialisation of European integration through the creation of ‘smart’ mobile Services that can be shared and used anywhere. The ambitions underpinning Citadel on the Move are (in line with its predecessor – the Citadel Statement – which aimed to pressure EU and National decisionmakers to help ‘make the Malmo Ministerial Declaration real at the local level) broad. If achieved, however, Citadel on the Move should provide the strongest proof yet that Open Data represents much, much more than an over-hyped bust.”

The article was originally published by EurActiv on 20/02/2012

Building the Evidence: Open Data and the Future of Construction

Who would use our Open Data? This bemused question from one of our Local Authority clients has always struck me as deceptively simple. On the one hand I can provide pat answers about the social value of Open data for transparency or the long-term benefits of a data-rich society but few of us are wired to work that way, least of all a busy executive trying to deliver a 21st century local government on a falling budget. What my client cared about, what most people care about, is the tangible, short-term benefits of Open Data. If Open Data can’t meet the ‘elevator pitch’ test of delivering an eye-catching impact in a short timeframe, we are not, as an industry, doing a good enough job of articulating the value.

The experience above is by no means unique to government. While our public authorities may be one of the largest data owners and often have the clearest imperative (we pay our taxes now give us our data!) to publish, the rising tide of this movement is now raising the ships of industry and third sector too. More and more businesses are waking up to the fact that releasing information gets a customer more engaged with your brand. Only one problem with this picture: Company data is just one side of the story. What about the information the consumer is feeding back to the company; shouldn’t that be open too?

Here Open Data blurs a line with crowdsourcing. By harvesting the opinions of hundreds even thousands of people about a company, a product or a campaign; you can gather hugely useful information. Combined with Open data from the company and the Government, these three sources have a supercharged capacity to generate new insights that can deliver economic value. So far, so abstract, so lets take the best example of an industry coming to grips with the power of Open data: the construction business.

Construction is, by its nature, a data-driven craft. From the exact measurements of a building to the numerous regulations that govern every aspect of a project, construction is a quantifiable business. The industry has also made giant leaps forward in recent years through the introduction of the Building Information Management Task Group. When 21c president Julia Glidden talked with Mark Bew, BIM’s Chairman, recently, what became clear was just how much work has gone into standardising the data of this industry. The rigorous models of a building BIM has created mean that everything in a neighbourhood or a city can be mapped, calibrated and explored using a single interface; a great leap forward for a data-driven industry.

Yet strangely, the construction industry has historically underweighted one of the key data sources in any building project: the way people respond to the finished product. When I say underweighted, I do not mean to suggest that painstaking work has not been done to survey and gather the views of people on a project; they most certainly have. But often, such surveys are qualitative, relatively small-scale and kept as closely held secrets by the companies who gather them. This approach makes little sense for two reasons: 1) The tools exist to gather large-scale quantitative evidence of people’s views and responses to building projects and 2) Construction is an industry where many companies work of similar types of projects, so collecting and sharing such data would be of mutual benefit.

So here is my suggestion to the construction industry: Create a system-level model of your next project, based on BIM standards, and then start filling in the gaps with Open Data from government and crowdsourced information from the general public.

The first of these points, Open Data from government, is easy. As an example, the Local Government Association’s recently concluded ‘Release of Data’ scheme got many local authorities to release their full planning application datasets. A construction company with access to this information could build a system for identifying where in the country similar projects have been successful and what the common characteristics of their applications have been. They can also do all this while slashing the overheads on lawyers and consultants needed to vet, shape and hone each application. The data provides a clear, evidence-based jumping off point for making smart planning applications. Also, government has a rich variety of information that augment a model. For example, if you knew detailed demographics about those surrounding a new development, you can make more informed predictions about how the community is likely to respond. An area of older, wealthy long-term homeowners may be more likely to favour low-rise developments that match the architecture of the local area, while 20-something professionals may be more receptive to a cutting-edge block. Simply by harvesting the government resources at their disposal, construction companies can get access to a trove of no-cost insights for their next project.

The second point, crowdsourced information from the public, is more of a challenge. What is needed is a system of capturing the response of people to their built environment and a way to feed the results into your next project. Thankfully a suite of techniques are helping to make this a reality. Social Media Analytics can provide a general sense of the reception of a discrete building or development by monitoring sentiment around mentions of the space. Likewise, advanced techniques like those being researched by Stuart Chalmers and the BRE are attaching headsets to people before walking them around an area to measure their emotional response to the architecture. Such techniques are staring to drive a revolution in Open Data and one which architecture can take full advantage of.

The best part you can explain this to anyone who asks in a single sentence: Open Data cuts your design costs, lowers consultation time and increases community acceptance of projects

21c is a specialist ICT innovation firm with an unparalleled track record helping companies and governments to use Open Data to deliver results.