Social media systems on the web have provided excellent platforms to facilitate and enable audience participation, engagement, and community, which has resulted in our new participatory culture. This blog explains What is Sentiment Analysis with examples.
From reviews and blogs to YouTube, Facebook, and Twitter, people have embraced these platforms enthusiastically because they enable their users to freely and conveniently voice their opinions and communicate their views on any subject across geographic and spatial boundaries.
They also allow people to easily connect with others and to share their information. This participatory web and communications revolution have transformed our everyday lives and society as a whole. It has also popularized two major research areas, namely, social network analysis and sentiment analysis.
Sentiment analysis essentially grew out of social media on the web that has been very active since the year 2002.
Apart from the availability of a large volume of opinion data in social media, opinions, and sentiments also have a very wide range of applications simply because opinions are central to almost all human activities. Whenever we need to make a decision, we often seek out others’ opinions.
Because a key function of social media is for people to express their views and opinions, sentiment analysis is right at the center of research and application of social media itself. It is now well recognized that to extract and exploit information in social media, sentiment analysis is a necessary technology.
One can even take a sentiment-centric view of social network analysis and, in turn, social media content analysis because the most important information that one wants to extract from the social network or social media content is what people talk about and what their opinions are. These are exactly the core tasks of sentiment analysis.
Sentiment Analysis and Natural Language Processing (NLP)
Sentiment analysis, also called opinion mining, is the field of study that analyzes people’s opinions, sentiments, appraisals, attitudes, and emotions toward entities and their attributes expressed in the written text. The entities can be products, services, organizations, individuals, events, issues, or topics.
The field represents a large problem space. The term opinion is taken to mean the whole concept of sentiment, evaluation, appraisal, or attitude and associated information, such as the opinion target and the person who holds the opinion, and the term sentiment is taken to mean the underlying positive or negative feeling implied by opinion.
Sentiment analysis or opinion mining aims to identify positive and negative opinions or sentiments expressed or implied in text and also the targets of these opinions or sentiments.
Sentiment analysis mainly focuses on opinions that express or imply positive or negative or sentiments, also called positive or negative or neutral opinions respectively in everyday language.
This type of opinion is similar to the concept of attitude in social psychology. Apart from sentiment and opinion, there are also concepts of affect, emotion, and mood, which are psychological states of mind.
Sentences expressing opinions or sentiments, being inherently subjective, are usually subjective sentences as opposed to objective sentences that state facts. However, objective sentences can imply positive or negative sentiments of their authors too, because they may describe desirable or undesirable facts.
Sentiment analysis is a semantic analysis problem, but it is highly focused and confined because a sentiment analysis system does not need to fully “understand” each sentence or document; it only needs to comprehend some aspects of it, for example, positive and negative opinions and their targets.
Owing to some special characteristics of sentiment analysis, it allows much deeper language analyses to be performed to gain better insights into NLP than in the general setting because the complexity of the general setting of NLP is simply overwhelming.
Although general natural language understanding is still far from us, with the concerted effort of researchers from different NLP areas, we may be able to solve the sentiment analysis problem, which, in turn, can give us critical insight into how to deal with general NLP.
The experience in the past 15 years seems to indicate that rather than being a subarea of NLP, sentiment analysis is actually more like a mini version of the full NLP or a special case of the full-fledged NLP;
The reason for this is that sentiment analysis touches every core area of NLP, such as lexical semantics, co-reference resolution, word sense disambiguation, discourse analysis, information extraction, and semantic analysis. Sentiment analysis is mainly carried out at three levels:
1.Document-level: Assuming that each document expresses opinions on a single entity (e.g., a single product or service), document-level sentiment classification indicates whether a whole opinion document expresses a positive or negative sentiment. For instance, given a product review, the system determines whether the review expresses an overall positive or negative opinion about the product.
2. Sentence level: Sentence-level sentiment classification indicates whether each sentence expresses a positive, negative, or neutral opinion. This level of analysis is closely related to subjectivity classification, which distinguishes sentences that express factual information (called objective sentences) from sentences that express subjective views and opinions (called subjective sentences).
3.Aspect level: If a sentence has multiple opinions, It does not make much sense to classify this sentence as positive or negative because it is positive about one entity but negative about another. To obtain this level of fine-grained results, we need to go to the aspect level.
Instead of looking at language units (documents, paragraphs, sentences, clauses, or phrases), aspect-level analysis directly looks at the opinion and its target (called opinion target).
Thus, the goal of this level of analysis is to discover sentiments on entities and/or their aspects. On the basis of this level of analysis, a summary of opinions about entities and their aspects can be produced.
There are two different types of opinions:
A regular opinion expresses a sentiment about a particular entity or an aspect of the entity, for example, “Orange tastes very good”express a positive sentiment or opinion on the aspect taste of orange. This is the most common type of opinion.
A comparative opinion compares multiple entities based on some of their shared aspects, for example, “Mango tastes better than orange” compares mango and orange based on their tastes (an aspect) and expresses a preference for mango.
Sentiment analysis involves addressing the problems of opinion searching and opinion summarization at appropriate levels.
Sentiment words, also called opinion words, are words in a language that indicate desirable or undesirable states. For example, good, great, and beautiful are positive sentiment words and bad, awful, and dreadful are negative sentiment words.
Sentiment words and phrases are instrumental in sentiment analysis. A list of such words and phrases is called a sentiment lexicon. Sentiment analysis is usually undertaken in the context of a predefined lexicon.
Individuals, organizations, and government agencies are increasingly using the content in social media for decision making. If an individual wants to buy a consumer product, he or she is no longer limited to asking his or her friends and family for opinions because there are many user reviews and discussions in public forums on the web about the product.
For an organization, it may no longer be necessary to conduct surveys, opinion polls, or focus groups to gather public or consumer opinions about the organization’s products and services because an abundance of such information is publicly available.
Governments can also easily obtain public opinions about their policies and measure the pulses of other nations simply by monitoring their social media.
Sentiment analysis applications have spread to almost every possible domain, from consumer products, healthcare, tourism, hospitality, and financial services to social events and political elections.
There are now hundreds of companies in this space, start-up companies and established large corporations, that have built or are in the process of building their own in-house capabilities, such as Google, Microsoft, Hewlett-Packard, Amazon, eBay, SAS, Oracle, Adobe, Bloomberg, and SAP.
A popular application of sentiment analysis is a stock market prediction. The system identifies opinions from message board posts by classifying each post into one of three sentiment classes: bullish (optimistic), bearish (pessimistic), or neutral (neither bullish nor bearish). The resulting sentiments across all stocks were then aggregated and used to predict the stock Index.
Instead of using bullish and bearish sentiments, an alternate approach is to identify positive and negative public moods on Twitter and used them to predict the movement of stock market indices such as the Dow Jones, S&P 500, and NASDAQ.
The analysis shows that when emotions on Twitter fly high, that is, when people express a lot of hope, fear, or worry, the Dow goes down the next day. When people have less hope, fear, or worry, the Dow goes up.
A mobile environment is different from traditional distributed environments due to its unique characteristics such as the mobility of users or computers, the limitation of computing capacity of mobile devices, and the frequent and unpredictable disconnection of wireless networks.
Therefore, the development of mobile systems is different from the development of distributed systems.
In other words, when designing a mobile system, we have to overcome challenges due to the physical mobility of the clients, the portability features of mobile devices and the fact that the communication is wireless. Thus, it is important that these issues are examined carefully when considering the system requirements, in terms of both functional and nonfunctional requirements.
The process to identify the requirements of a mobile client-server-based system is very different from a nonmobile one. This is due to the unique characteristics of mobile environments that are the mobility of users or computers, the limitation of the computing capacity of mobile devices, and the frequent and unpredictable disconnections of wireless networks.
Mobile Computing Applications
A wireless mobile application is defined as a software application, a wireless service, or a mobile service that can be either pushed to users’ handheld wireless devices or downloaded and installed, over the air, on these devices.
Wireless applications themselves can be classified into three streams:
1. Browser-based: Applications developed using a markup language. This is similar to the current desktop browser model where the device is equipped with a browser. The wireless application protocol (WAP) follows this approach.
2. Native applications: Compiled applications where the device has a runtime environment to execute applications. Highly interactive wireless applications are only possible with the latter model.
Interactive applications, such as mobile computer games, are a good example. Such applications can be developed using the fast-growing Java 2 Micro Edition (J2ME) platform, and they are known as MIDlets.
3. Hybrid applications: Applications that aim at incorporating the best aspects of both streams above: the browser is used to allow the user to enter URLs to download native applications from remote servers, and the runtime environment is used to let these applications run on the device.
Mobile Text Analysis
Another technology that can be used for data mining mobile devices is text mining, which refers to the process of deriving, extracting, and organizing high-quality information from unstructured content, such as texts, emails, documents, messages, and comments.
Text mining means extracting meaning from social media and customer comments about a brand or company in mobile sites and app reviews.
This is a different variation in clustering programs; text mining software is commonly used to sort through unstructured content that can reside in millions of emails, chat, web forums, texts, tweets, blogs, and so on, that daily and continuously accumulate in mobile sites and mobile servers.
Text analytics generally includes tasks such as the following:
Categorization of taxonomies
Clustering of concepts
Entity and information extraction
Text analytics is important to the data mining of mobile devices because, increasingly, companies, networks, mobile sites, enterprises, and app servers are accumulating a large percentage of their data in unstructured formats, which is impossible to analyze and categorize manually.
Text mining refers to the process of deriving an understanding from unstructured content through the division of clustering patterns and the extraction of categories or mobile trends using machine learning algorithms for the organization of key concepts from unstructured content.
Text mining can be used to gain new insight into unstructured content from multiple data sources, such as a social network of a mobile site or an app platform. Text analytical tools can convert unstructured content and parse it over to a structured format that is amenable to data mining of mobile devices via classification software.
For example, all the daily emails or visits that a mobile site accumulates on a daily basis can be organized into several groupings, such as those mobiles seeking information, service assistance, or those complaining about specific products, services, or brands. Text mining can also be used to gauge sentiment regarding a brand or company.
Mobile marketers, developers, and brands need to consider how to incorporate time, demographics, location, interests, and other mobile available variables into their analytics models. Clustering, text, and classification software can be used to accomplish this for various marketing and brand goals.
Clustering software analyses can be used to discover and monetize mobile mobs. Text software analyses can discover the important brand value and sentiment information being bantered about in social networks.
Finally, classification software can pinpoint important attributes of profitable and loyal mobiles. Classification often involves the use of rule-generating decision-tree programs for the segmentation of mobile data behaviors.
Mobile Classification Analysis
There are two major objectives to classification via the data mining of mobile devices: description and prediction. The description is an understanding of a pattern of mobiles behaviors and to gain insight—for example, what devices are the most profitable to a mobile site and app developer.
Prediction, however, is the creation of models to support, improve, and automate decision makings, such as what highly profitable mobiles to target in an ad marketing campaign via a mobile site or app.
Both description and prediction can be accomplished using classification software, such as rule-generator and decision-tree programs. This type of data mining analysis is also known as supervised learning.
For example, a mobile analyst or marketer can take advantage of segmenting the key characteristics of mobile behaviors over time to discover hidden trends and patterns of purchasing behaviors.
Machine learning technology can discover the core features of mobiles by automatically learning to recognize complex patterns and make intelligent decisions based on mobile data, such as what, when, where, and why certain mobiles have a propensity to make a purchase or download an app, while others do not.
Classifying mobiles enables the positioning of the right product, service, or content to these moving devices via precise messages on a mobile site, or the targeting of an email, text, or the creation of key features to an app.
The marketer or developer will need to use classification software known as rule-generators or decision-tree programs. Decision trees are powerful classification and segmentation programs that use a tree-like graph of decisions and their possible consequences.
Decision-tree programs provide a descriptive means of calculating conditional probabilities. Trained with historical data samples, these classification programs can be used to predict future mobile behaviors.
A decision tree takes as input an objective, such as what type of app to offer, described by a set of properties from historical mobile behaviors or conditions, such as geolocation, operating system, and device model. These mobile features can then be used to make a prediction, such as what type of app to offer to a specific mobile.
The prediction can also be a continuous value, such as total expected coupon sales, or what price to offer for an app.
When a developer or marketer needs to make a decision based on several consumer factors, such as their location, device being used and total log-in time.
A decision tree can help identify which factors to consider and how each factor has historically been associated with different outcomes of that decision—such as what products or services certain mobiles are likely to purchase based on observed behavioral patterns over time.
One common advantage of using decision trees is to eliminate a high number of noisy and ineffective consumer attributes for predicting, say, “high customer loyalty” or “likely to buy” models.
Developers and marketers can start with hundreds of mobile attributes from multiple data sources and, through the use of decision trees, they can eliminate many of them in order to focus simply on those with the highest information gain as they pertain to predicting high loyalty or potential revenue growth from mobile features and behaviors.
Mobile Streaming Analysis
The data mining of mobile devices may require the use of both deductive and inductive “streaming analytical” software that is event-driven to link, monitor, and analyze mobile behaviors. These new streaming analytical software products react to mobile consumer events in real time. There are two main types of streaming analytical products:
1. There are deductive streaming programs that operate based on user-defined business rules and are used to monitor multiple streams of data, reacting to consumer events as they take place.
2. There are also inductive streaming software products that use predictive rules derived from the data itself via clustering, text, and classification algorithms. These inductive streaming products build their rules from global models involving the segmentation and analysis from multiple and distributed mobile data clouds and networks.
These deductive and inductive software products can work with different data formats, from different locations, to make real-time predictions using multiple models from massive digital data streams.
J2EE is the result of Sun’s effort to integrate the assortment of Java technologies and API together into a cohesive Java development platform for developing complex distributed Java applications.
Sun’s enhancement of the n-tier development model for Java, combined with the introduction of specific functionalities to permit the easier development of the server- side scalable Web-based enterprise applications, has led to wide adoption of Java for Web-centric application development.
Enterprise application development entails expertise in a host of areas like interprocess communications, memory management, security issues, and database-specific access queries. J2EE provides built-in support for services in all these areas, enabling developers to focus on implementing business logic rather than intricate code that supports basic application support infrastructure.
There are numerous advantages of application development in the J2EE area:
J2EE offers support for the componentization of enterprise applications that enable higher productivity via reusability of components, the rapid development of functioning applications via prebuilt functional components, higher-quality test-driven development via pretested components, and easier maintenance via cost-effective upgrades to individual components.
J2EE offers support for hardware and operating systems (OS) independence by enabling system services to be accessed via Java and J2EE rather than directly via APIs specific to the underlying systems.
J2EE offers a wide range of APIs to access and integrate with third-party products in a consistent manner, including databases, mail systems, and messaging platforms.
J2EE offers clear-cut segregation between system development, deployment, and execution, thus enabling independent development, integration, and upgradation of components.
J2EE offers specialized components that are optimized for specific types of roles in an enterprise application such as Entity Beans for handling persistent data and Session Beans for handling processing.
All the aforementioned features make possible rapid development of complex, distributed applications by enabling developers to focus on developing business logic, implementing the system without being impacted by prior knowledge of the target execution environment(s), and creating systems that can be ported more easily between different hardware platforms and operating systems (OS).