A Data Platform for Intelligent Systems

Internet of Things

A key driver in the development of smart environments is the convergence of technologies such as the IoT and big data, which is driving the digitisation of physical infrastructures with sensors, networks and social capabilities (Sheth, Anantharam and Henson, 2013). The vision of the IoT is a complicated proposition that requires end-to-end distributed systems from the development of new electronic devices and embedded systems, new forms of data processing to deal with the volume, variety, and velocity of data generated, to enhanced user experiences leveraging cognitive and behavioural models with new data visualisation and interaction paradigms.

As the IoT is enabling the deployment of lower-cost sensors, we see more broad adoption of IoT devices/sensors and gain more visibility (and data) into smart environments. This results in large quantities of high volume and high-velocity event streams from smart environments that need to be processed. IoT-based smart environments are also generating different types of data with an increase in the number of multimedia devices deployed, such as vehicle and traffic cameras. The IoT is driving the deployment of intelligent systems and creating new opportunity in smart environments:

  • Digital Twins: A digital replica of physical assets (car), processes (value-chain), systems, or physical environments (building). The digital representation (i.e. simulation modelling or data-driven models) provided by the digital twin can be analysed to optimise the operation of the “physical twin”.
  • Physical-Cyber-Social (PCS): A computing paradigm that supports a richer human experience with a holistic data-rich view of the smart environment that integrates, correlates, interprets, and provides contextually relevant abstractions to humans (Sheth, Anantharam and Henson, 2013).
  • Mass Personalisation: More human-centric thinking in the design of systems where users have growing expectations for highly personalised digital services for the “Market of One”.
  • Data Network Effects: As more systems/users join and contribute data to the smart environment, a “network effect” can take place, resulting in the overall data available becoming more valuable.

Within this context, we are interested in how data created within a smart environment can be leveraged by intelligent systems, and how data can be easily shared within the ecosystem of systems (new and old) and stakeholders.

Data Ecosystems

A Data Ecosystem is a socio-technical system enabling value to be extracted from data value chains supported by interacting organisations and individuals (Curry, 2016). Within an ecosystem, data value chains are oriented to business and societal purposes. The ecosystem can create the conditions for a marketplace competition among participants or enable collaboration among diverse, interconnected participants that depend on each other for their mutual benefit.

The digital transformation is creating a data ecosystem with data on every aspect of our world, spread across a range of intelligent systems. As illustrated below, a smart environment enabled with IoT data, and contextual data sources, results in a data-rich ecosystem of structured and unstructured data (e.g. images, video, audio, and text) that can be exploited by data-driven intelligent systems.

There is a need to bring together data from the multiple intelligent systems that exist within the data ecosystem that surrounds a smart environment. For example, smart cities are showing how different systems within the city (e.g. energy and transport) can collaborate to maximise the potential to optimise overall city operations. At the level of an individual, digital services can deliver a personalised and seamless user experience by bringing together relevant user data from multiple systems (Curry et al., 2018). This requires a System of Systems (SoS) approach to connect systems that cross organisational boundaries, come from various domains, (e.g. finance, manufacturing, facilities, IT, water, traffic, and waste) and operate at different levels (e.g. region, district, neighbourhood, building, business function, individual).

Data ecosystems present new challenges to the design of intelligent systems and SoS that require a rethink in how we should deal with the needs of large-scale data-rich smart environments. How can we support data sharing between intelligent systems in a data ecosystem? What are the technical and non-technical barriers to data sharing within the ecosystem? How can intelligent systems leverage their data ecosystem to be “smarter”? Solving these problems is critical if we are to maximise the potential of data-intensive intelligent systems (Curry and Sheth, 2018).

Enabling Data Ecosystem for Intelligent Systems

A screenshot of a cell phone

Description automatically generated
Four-layered framework to enable data ecosystems for intelligent systems within IoT-based smart environments (Curry et al., 2019)

There is a clear need to support knowledge sharing among intelligent systems within a smart environment. Understanding the data management challenges in more detail requires an appreciation of how the IoT is enabling smart environments. The range of IoT challenges can be studied based on the three-layered framework by Atzori et al. (Atzori, Iera and Morabito, 2010) Layer 1 – Communication and Sensing, Layer 2 – Middleware, and Layer 3- Users, Applications, & Analytics. While significant efforts in the area of IoT come from the communication and networking levels, there has been a growing realisation that the challenges of the IoT will be more prevalent at the data layer (Aggarwal, Ashish and Sheth, 2014) including data collection, management, analytics, and sharing. At the data-level, intelligent systems can benefit from leveraging data from multiple systems within the smart environment. However, many of the data management and sharing activities are currently performed at the application layer within IoT deployments. To capture these data management and sharing activities, we have elaborated a four-layered framework for enabling data ecosystems for intelligent systems within IoT-based smart environments, which builds on the work by Atzori et al. (Atzori, Iera and Morabito, 2010). As illustrated in Fig 1.2, we introduce a 4th Layer between the Middleware and Application layers to support data management and sharing activities. The four-layered framework for enabling data ecosystems for intelligent systems consists of:

  • Layer 1- Communication and Sensing: An essential requirement is an infrastructure of communication and sensing that maps the world of physical things into the world of computationally processable data.
  • Layer 2- Middleware: There is a need for middleware that can abstract the application developers from the underlying technologies. Data distribution, processing, and access to legacy information systems take place at this layer.
  • Layer 3- Data: There is a need to enable data management and sharing activities, including managing schema and entities, accessibility, access control, data quality, and licensing take place at this layer.
  • Layer 4- Intelligent Applications, Analytics, and Users: Users expect IoT-based analytics and applications that present the data gathered and analysed in an intuitive and user-friendly manner using new visualisations and user experiences to ensure cognitive-friendly smart environments.

Our key addition is Layer 3- Data, which requires the development of data infrastructure to support the sharing and management of data among systems in the ecosystem. Platform approaches have proved successful in many areas of technology, and the idea of large-scale “data” platforms are touted as a possible next step. A data platform focuses on the secure and trusted data sharing among a group of participants (e.g. industrial consortiums sharing private or commercially sensitive data) within a clear legal framework. Within a smart environment, a data platform would have support continuous, coordinated data flows, seamlessly moving data among intelligent systems.

Excerpt from: Curry E. (2020) Real-time Linked Dataspaces: A Data Platform for Intelligent Systems Within Internet of Things-Based Smart Environments. In: Real-time Linked Dataspaces. Springer, Cham.


Aggarwal, C. C., Ashish, N. and Sheth, A. (2014) ‘The internet of things: A survey from the data-centric perspective’, in Managing and Mining Sensor Data, pp. 383–428. doi: 10.1007/978-1-4614-6309-2_12.

Atzori, L., Iera, A. and Morabito, G. (2010) ‘The Internet of Things: A survey’, Computer Networks, 54(15), pp. 2787–2805. doi: 10.1016/j.comnet.2010.05.010.

Curry, E. (2016) ‘The Big Data Value Chain: Definitions, Concepts, and Theoretical Approaches’, in

Cavanillas, J. M., Curry, E., and Wahlster, W. (eds) New Horizons for a Data-Driven Economy: A Roadmap for Usage and Exploitation of Big Data in Europe. Springer International Publishing. doi: 10.1007/978-3-319-21569-3_3.

Curry, E. et al. (2018) ‘Internet of Things Enhanced User Experience for Smart Water and Energy Management’, IEEE Internet Computing, 22(1), pp. 18–28. doi: 10.1109/MIC.2018.011581514.

Curry, E. et al. (2019) ‘A Real-time Linked Dataspace for the Internet of Things: Enabling “Pay-As-You-Go” Data Management in Smart Environments’, Future Generation Computer Systems, 90, pp. 405–422. doi: 10.1016/j.future.2018.07.019.

Curry, E. and Sheth, A. (2018) ‘Next-Generation Smart Environments: From System of Systems to Data Ecosystems’, IEEE Intelligent Systems, 33(3), pp. 69–76. doi: 10.1109/MIS.2018.033001418.

Sheth, A., Anantharam, P. and Henson, C. (2013) ‘Physical-cyber-social computing: An early 21st century approach’, IEEE Intelligent Systems, 28(1), pp. 78–82. doi: 10.1109/MIS.2013.20.