Data drives the modern organizations of the world and hence making sense of this data and
unraveling the various patterns and revealing unseen connections within the
vast sea of data becomes critical and a hugely rewarding endeavor indeed. There
is a need to convert Big Data into Business
Intelligence that enterprises can readily deploy. Better data
leads to better decision making and an improved way to strategize for
organizations regardless of their size, geography, market share, customer
segmentation and such other categorizations.
is the platform of choice for working with extremely
large volumes of data. The most successful enterprises of tomorrow will be the
ones that can make sense of all that data
at extremely high volumes and speeds in order to capture
newer markets and customer base.
5 V’S OF BIG DATA:-
Big Data has certain characteristics and hence is defined using
Volume: The amount of data that businesses can collect is
really enormous and hence the volume of the data becomes a critical factor in Big Data analytics.
Velocity: The rate at which new data is being generated all
thanks to our dependence on the internet, sensors, and machine-to-machine data
is also important to parse Big Data in a timely manner.
data that is generated is completely heterogeneous in the sense that it could
be in various formats like video, text, database, numeric, sensor data and so
on and hence understanding the type of Big Data is a key factor to unlocking its value.
whether the data that is available is coming from a credible source is of
utmost importance before deciphering and implementing Big Data for business
Value: Last but not least, big data must have
value. That is, if you’re going to invest in the infrastructure required to
collect and interpret data on a system-wide scale, it’s important to ensure
that the insights that are generated are based on accurate data and lead to
measurable improvements at the end of the day.
data is an evolving term that describes any voluminous amount of
structured, semi structured and unstructured data that has the
potential to be mined for information.
Big data technologies are important in providing more
accurate analysis, which may lead to more concrete decision-making resulting in
greater operational efficiencies, cost reductions, and reduced risks for the
To harness the power of big data, you would require an
infrastructure that can manage and process huge volumes of structured and
unstructured data in realtime and can protect data privacy and security.
Some technologies used for big data
1. Hadoop: – Hadoop is an Apache open source
framework written in java that allows distributed processing of large datasets
across clusters of computers using simple programming models. A Hadoop
frame-worked application works in an environment that provides distributed
storage and computation across clusters of computers. Hadoop is designed to
scale up from single server to thousands of machines, each offering local
computation and storage.
2. MongoDB: – MongoDB
is an open-source document database that provides high
performance, high availability, and automatic scaling. MongoDB obviates the
need for an Object Relational Mapping (ORM) to facilitate development.
3. MapReduce: – MapReduce is a programming model for writing applications that
can process Big Data in parallel on multiple nodes. MapReduce provides
analytical capabilities for analyzing huge volumes of complex data. MapReduce
divides a task into small parts and assigns them to many computers. Later, the
results are collected at one place and integrated to form the result dataset.
4. Hive:- Hive
is a data warehouse infrastructure tool to process structured data in Hadoop.
It resides on top of Hadoop to summarize Big Data, and makes querying and
analyzing easy. Initially Hive was developed by Facebook, later the Apache
Software Foundation took it up and developed it further as an open source under
the name Apache Hive. It is used by different companies. For example, Amazon
uses it in Amazon Elastic MapReduce.
5. Apache Pig:- Apache Pig is an abstraction over MapReduce. It is a
tool/platform which is used to analyze larger sets of data representing them as
data flows. Pig is generally used with Hadoop; we can perform all the data
manipulation operations in Hadoop using Pig.
WHAT IS A
city is an urban area that uses different types of electronic
data collection sensors to supply information used to manage assets and
resources efficiently. This includes data collected from citizens,
devices, and assets that is processed and analyzed to monitor and manage
traffic and transportation systems, power plants, water supply networks, waste
management, law enforcement, information systems, schools, libraries,
hospitals, and other community services.
city concept integrates information and
communication technology (ICT),
and various physical devices connected to the network (the Internet of things or IoT) to optimize the
efficiency of city operations and services and connect to citizens. Smart
city technology allows city officials to interact directly with both community
and city infrastructure and to monitor what is happening in the city and how
the city is evolving.
of smarter cities:-
Environmental sustainability and efficiency
Sustainable homes and buildings
Efficient use of resources
Efficient and sustainable transportation
urban planning – livable cities
Monitoring of parking spaces availability in the city.
Monitoring of vibrations and material conditions in buildings, Bridges and
Noise Urban maps:
Sound monitoring in bar areas and centric zones in real time.
Detect smart phones and in general any device which works with Wifi or
Electromagnetic field levels: Measurement of the energy radiated by cell stations and WiFi
Monitoring of vehicles and pedestrian levels to optimize driving and walking
Intelligent and weather adaptive lighting in street lights.
Detection of rubbish levels in containers to optimize the trash collection
roads: Intelligent Highways with warning
messages and diversions according to climate conditions and unexpected
events like accidents or traffic jams.