The Opinion Mining Architecture OMA

1 downloads 0 Views 4MB Size Report
In Fred, A., Filipe, J., eds. : IC3K17 - Proceedings of the ...... Chajewska, U, Koller, D & Parr, R 2000, 'Making Rational Decisions using Adaptive Utility. Elicitation' ...
Working & Discussion Paper No 3│2017

---------------------------------------------------------------------

The Opinion Mining Architecture OMA Foundations of OMA in Machine Learning, Natural Language Processing and Description Logics --------------------------------------------------------------------Klemens Schnattinger & Heike Walterscheid ---------------------------------------------------------------------

Herausgeberin Duale Hochschule Baden-Württemberg Lörrach Prof. Dr. habil. Heike Walterscheid Hangstraße 46-50 │ DE-79539 Lörrach www.dhbw-loerrach.de ISSN: 2196-8162 ISSN: 2568-2288 (online)

Vorwort der Herausgeberin dieser Ausgabe Digitalisierung ist DAS Schlagwort unserer Zeit. Als Grundlage der Digitalisierung kommen u.a. Methoden aus der Künstlichen Intelligenz (KI) zum Einsatz. Da über 80% aller Informationen in Textform vorliegen, liegt nahe, die in diesen natürlich-sprachlichen Texten kodierte Information zu heben. Die dafür infrage kommenden Methoden der KI stammen aus dem Teildisziplinen des Natural Language Processing (NLP) und des Machine Learning (ML). In dem vorliegenden Beitrag wird eine allgemein verwendbare Software-Architektur (OMA = Opinion Mining Architecture) zur Analyse von Texten (Tweets) vorgestellt, die mit Methoden

der

oben

genannten

Teildisziplinen

Sentiment-Scores

zu

beliebigen

Betrachtungsgegenständen vollständig automatisch berechnen. Diese Anwendung wurde in einem ersten Szenario angewendet auf die Stimmungen zu den Finanzinstituten Deutsche Bank, Commerzbank, Volksbank und Sparkasse im Betrachtungszeitraum 15. Januar 2017 bis 13. Mai 2017. Im März 2017 haben zwei der betrachteten Banken Einführung von Kontoführungsgebühren angekündigt und im April 2017 dann diese Gebühren bereits erhoben. Den daraus resultierenden Stimmungsumschwung konnte mit OMA eindrücklich gezeigt werden. Mit dem vorgeschlagenen Methoden-Mix aus NLP und ML sind neben den Stimmungsmessungen aus Textquellen auch andere Anwendungen möglich: Berechnung von neuen, bisher unbekannten Informationen aus Texten (Knowledge Acquisition) wie z.B. das Lernen von Regeln aus Reports in unterschiedlichen Kontexten, das automatische Erweitern von Produktdatenbanken aus Internettexten, aber auch das Bestimmen von Präferenz zur Entscheidungsunterstützung in Kontexten von Politik, Wirtschaft und Gesellschaft. Die Ergebnisse aus der vorgestellten Analysemethode können im Rahmen von strategischen Unternehmensentscheidungen einen wertvollen Beitrag leisten. Das hier vorgestellte Paper ist ein erstes Ergebnis aus einem, von der Dr. Karl Helmut Eberle Stiftung geförderten, Forschungsprojektes an der DHBW Lörrach und wurde in einer kürzeren Fassung als Konferenzbeitrag (KDIR)1 angenommen und präsentiert. Lörrach, November 2017

1

Heike Walterscheid

Opinion Mining Meets Decision Making: Towards Opinion Engineering. In Fred, A., Filipe, J., eds. : IC3K17 - Proceedings of the. 9th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management, Funchal, Madeira, Portugal, vol. Volume 1: KDIR, pp.334-341.

2

The Opinion Mining Architecture OMA* Foundations of OMA in Machine Learning, Natural Language Processing and Description Logics Klemens Schnattinger and Heike Walterscheid Duale Hochschule Baden-Württemberg Lörrach

Abstract: This article introduces the basics of Opinion Mining Architecture OMA. On the one hand, these theories and methods from machine learning, natural language processing and weighted descriptive logic are presented. On the other hand, a first evaluation of OMA is shown using the example of tweets to analyze the sentiments of banks.

Keywords: Natural Language Processing, Machine Learning, Description Logics, Opinion Mining, Sentiment Analysis, Decision Making, Utility Theory

_______________________ * Basic long version paper of an accepted Paper at the KDIR 2017.

3

Contents 1.

Introduction ....................................................................................................................... 5 

2.

Foundations of Opinion Mining ....................................................................................... 6  2.1  Opinion und Opinion Mining .................................................................................... 6  2.2  NLP Techniques for Opinion Mining ........................................................................ 7  2.3  Machine Learning for Opinion Mining ..................................................................... 9  2.4  Comparative Opinion Mining .................................................................................. 12  2.5  Deep Learning for Opinion Mining ......................................................................... 13 

3.

Foundations of Decision Making .................................................................................... 14  3.1  Preference and Utility .............................................................................................. 14  3.2  Description Logics ................................................................................................... 16  3.3  Weighted Description Logics .................................................................................. 17 

4.

OMA – An Architecture for Opinion Mining ................................................................. 21 

5.

Conclusion ...................................................................................................................... 23 

References ................................................................................................................................ 25 

4

1. Introduction Opinion mining (Pang & Lee 2008) as well as decision making (Keeney & Raiffa 1993) executed by intelligent, autonomous agents resound throughout the land. The work reported in this paper is part of the project OMA aiming at the development of an opinion mining and evaluation system for real-world domains like product pricing (Archak, Ghose & Ipeirotis 2007), market prediction (Zhang & Skiena 2010), election forecasting (O'Connor et al. 2010), nation relationship analysis (Chambers et al. 2015), and risk detection in the financial sector (Nopp & Hanbury 2015). The methodological challenge is two-fold. The opinion mining task is that the textual sources must be preprocessed and analyzed at different depths. On the other hand, the opinion evaluation task is to put the mined opinions/sentiments in an order resp. preference relation. To address these problems, we first introduce an evaluation of natural language processing (NLP) and integrated machine learning (ML) techniques for opinion mining (Sun, Luo & Chen 2017). Particularly, we discuss general NLP techniques that are commonly used for preprocessing texts, approaches of opinion mining at different levels and situations, comparative opinions, and deep learning approaches. Second, we propose a theoretic framework to map decision making problems to a weighted extension of description logics (Acar et al. 2017). Particularly, we propose to address this task with decision making systems which are initially based on multi-attribute utility theory (MAUT) (Keeney & Raiffa 1993). Since then various approaches have emerged. Among others a popular approach is the application of logic for decision and utility theoretical problems (Schnattinger & Hahn 1998), (Lafage & Lang 2000), (Chevaleyre, Endriss & Lang 2006). Hence, we bring together NLP and ML techniques for opinion mining and weighted description logics for decision making in a common architecture, called OMA, the Opinion Mining Architecture (cf. section 4). The tasks, the theoretical approaches and the sources as well as their assigned actions of both the Opinion Mining and Sentiment Analysis as well as the Decision Making and Utility Theory are shown in Table 1 on the next page.

5

Opinion Mining &

Tasks

Decision Making &

Sentiment Analysis

Approaches

NLP & ML

Sources&

Texts: preprocess,

actions

analyze, extract

Utility Theory Weighted Description Logics Sentiments & Opinions: evaluate

Table 1: Summary of the tasks, approaches and related sources

2. Foundations of Opinion Mining 2.1 Opinion und Opinion Mining Usually, the term opinion is defined as “the personal view that someone has about something” (Dictionary.com 2002). Synonyms are among others assessment, assumption, attitude, conclusion, feeling, idea, impression, judgment, point of view, reaction, sentiment, speculation, thought. Formally, an opinion is defined as follows due to (Liu 2012): ,

,

, ,

, where

th opinion holder,

denotes the th entity,

the th aspect of the th entity,

the time when the opinion is expressed,

towards the th aspect of the th entity from opinion holder

the opinion or sentiment

at time .

For example, in the statement “The screen of this tablet is good”, the components and

the

,

can be identified: screen is an aspect of entity tablet. Additionally, a positive

sentiment is expressed. The opinion holder and the time are not given. As we can see all five components are not always necessary to express an opinion. The three mentioned components are adequate for document level opinion mining. In contrast, further components are needed for fine-grained opining mining tasks, such as summarization (as we will see later). To perform opinion mining, machine learning approaches are meaningful. Commonly, classification is used to accomplish opinion mining for a whole document (so-called document level opinion mining) or also for a sentence (so-called sentence level opinion mining). The training of the classifiers is carried out using known texts in order to identify the sentiment orientation of the available texts. Among others Naïve Bayes classifier can be used. However, a classifier which is trained from one domain misfits in another domain (so-called cross-domain problem). The problems become more fatal in the cross-lingual situation. Applying a trained classifier to texts in another language leads to severe problems (so-called cross-lingual problem). For the task of identifying the opinion holder, detecting opinion expressions, and identifying the target or 6

aspect of the opinion (so-called fine-grained opinion mining), corpora with annotated opinion or sentiment scores are necessary but difficult to get. In contrast to the supervised methods, lexicon approaches identify the sentiment score of text purely without a training set (unsupervised) according to given sentiment lexicons. In this context, a sentiment lexicon is a dictionary of sentiment words and phrases. Additionally, it contains a sentiment orientation and a strength for each sentiment entry. The orientation for each document or sentence is expressed through a sentiment score. This score is computed by the sentiment orientation and strengths of the words or phrases in the considered text or sentence. Lexicon approaches have the advantage of using less resources, because they don’t use annotated corpora. In addition, such a sentiment lexicon can be integrated into machine learning approaches and so, performance can be significantly increased.

2.2 NLP Techniques for Opinion Mining To perform opinion mining the reviewed texts must be preprocessed. For this purpose, the following processes are usually carried out for structuring the text and for extracting features: tokenization, Part of Speech (POS) tagging and parsing. Tokenization decomposes a sentence or document into tokens. Tokens represents words or phrases. For English or German, the decomposition of words is easy with the spaces, but some additional expertise should be kept in mind, such as opinion phrases and named entities. Words, such as “the”, “a” only provide little information. Thus, tokenization must remove these words, which are called stop words. Many tokenization tools are available (see tools below). For Chinese or other languages without explicit markers for word boundaries, tokenization is not trivial. Word segmentation must be used for this purpose. We will not continue to look at word segmentation for these languages. POS tagging is a technique that analyzes the lexical information of a word for determining the corresponding POS tag (e.g. adjective or noun). POS tagging is a so-called sequential labeling problem. Conditional Random Fields (CRFs) (Lafferty, McCallum & Pereira 2001) and outperform hidden as well as maximum-entropy Markov models (Sutton & McCallum 2011) are commonly applied to this problem. The POS tags are quite helpful. On the one hand, adjectives can represent opinion words. In the other hand, entities and aspects of opinion mining can be expressed with nouns or combination of nouns. Parsing is a technique that provides syntactic information. Among other things, it analyses the grammatical structures of a given sentence and generates a tree with the corresponding

7

relationship of different so-called constituents. A constituent is “a group of words treated by a syntactic rule as a unit” (Carnie 2010). Unlike POS tagging, parsing determines richer structural information. It can be used especially for fine-grained opinion mining (e.g. (Socher et al. 2013)). NLTK2   The Natural Language Toolkit (NLTK) is an open source platform for NLP tasks like tokenization, stemming, POS tagging, parsing, and semantic reasoning (Bird, Klein & Loper 2009). It also provides interfaces for corpora and lexicons. In the POS tagging module, several common taggers are provided. In named entity recognition module, a maximum entropy classifier is taken to pre-trained models for cities, states/provinces, and countries. In the parsing module, different approaches are also pursued. NLTK also supports semantic reasoning in first order logics. The most of pre-trained models in NLTK are limited on English. However, training APIs for other languages are provided. Its programming language is Python. OpenNLP3   Apache OpenNLP is a Java library for processing natural language texts and includes tokenization, sentence segmentation, POS tagging, named entity recognition, parsing, and coreference resolution (Reese 2015). For named entity recognition, POS tagging, chunking and coreference resolution maximum entropy classifiers are implemented. A chunking parser is used in the parsing module. As well as in NLTK the most of pre-trained models in OpenNLP are limited on English, but training APIs to train other languages are provided. CoreNLP4   Stanford CoreNLP is a framework which also supports POS tagging, named entity recognition, parsing, and coreference resolution (Manning et al. 2014), (Reese 2015). It also offers advanced sentiment analysis features (Socher et al. 2013) based on CRFs, maximum entropy models and deep learning. Stanford CoreNLP supports as languages among others Arabic, Chinese, French, Spanish and German. It also offers sentiment analysis based on deep learning approaches for English. Its programming language is Java.

2 3 4

http://www.nltk.org http://opennlp.apache.org http://stanfordnlp.github.io/CoreNLP

8

Gensim5   Gensim is an open source library for topic modeling (Řehůřek & Sojka 2010). Topic modeling involves discovering the abstract themes that occur in (a collection of) documents. Among others it includes online Latent Semantic Analysis (LSA), Latent Dirichlet Allocation (LDA) and Hierarchical Dirichlet Process. Its programming language is Python.

2.3 Machine Learning for Opinion Mining For opinion mining gaining features from texts is very important. Thus, text features are discussed, including n-gram features with information retrieval (IR) weighting schemes, syntactic features and semantic features. An n-gram is a set of n adjacent items. Additionally, the number of times an item appears in the text is denoted. In opinion mining, double-digit weights of unigram (1-gram) and bigram (2-gram) are widely accepted. Instead of binary weights, other IR weighting schemes can be used (Paltoglou & Thelwall 2010). Syntactic features include POS tags as well as syntactic information. These features either build up a so-called feature space for machine learning approaches (Gamon 2004), (Joshi & Penstein-Rosé 2009), or generate rules for e.g. entities and aspects in fine-grained opinion mining (Gindl, Weichselbraun & Scharl 2013). Semantic features are conjunctions which specifies negation, increase, and decrease of a perception. Negation turns the sentiment orientation into the opposite. Increase and decrease also influence the strength of sentiment, respectively, are useful for opinion mining (Choi & Cardie 2008), (Kennedy & Inkpen 2006), (Taboada et al. 2011). Opinion mining is usually divided into three levels: document level, sentence level, and fine-grained level. The literature also discusses the concepts of cross-domain and crosslingual opinion mining. In the following, these five terms of opinion mining will be presented in more detail. The task of the document level opinion mining determines sentiment orientation of an entire document, such as reviews of movies or products, tweets and blogs. The objective of the document level opinion is identifying the fifth component of the quintuple from above, that is the

5

in

,

,

, ,

.

http://radimrehurek.com/gensim

9

Recent techniques for document level opinion mining are among others: 

Supervised approaches Usual classifiers in machine learning, such as a Naïve Bayes classifier, are used. The features considered are, among others, n-gram, POS tags, position information (Pang, Lee & Vaithyanathan 2002), semantic features (Kennedy & Inkpen 2006) and discourse features (Somasundaran et al. 2009). There also exists common classifiers based on SVM and Naïve Bayes classifier (e.g. (Wang & Manning 2012)).



Probabilistic generative model based approaches Generative models such as joint sentiment topic model (Lin & He 2009) are used. The transitions between sentiments of words are modelled with a Markov chain.



Unsupervised lexicon-based approaches Averaged sentiment orientation is used to suggest the overall sentiment orientation of an entire document (Turney 2002). To improve the results e.g. tightening and negation indicators (Taboada et al. 2011) as well as discourse structure-based weighting scheme (Bhatia, Ji & Eisenstein 2015) are proposed.

In opinion mining at the sentence level, sentiment orientation is determined for each sentence in the document. However, on the sentence level, not all the detailed information of opinions is collected such as opinion target and opinion holder. For example, “The screen of this tablet is good.” expresses a positive sentiment orientation to aspect “screen” of entity “tablet”. Recent techniques for sentence level opinion mining are among others: 

Supervised approaches Again, Naïve Bayes classifiers are used to determine subjectivity of sentences (Yu & Hatzivassiloglou 2003). CRFs are, however, suggested for the dependencies between sentences. (Yang & Cardie 2015). There also exists a common segmentation and classification framework (Tang et al. 2015).



Unsupervised approaches For subjectivity classification in sentences graph-based approaches exists (Pang & Lee 2004), also, a lexicon-based approach (Kim & Hovy 2004).

The problems with the fine-grained level opinion mining can no longer be traced with traditional classification techniques. It is suggested to discover opinion details in texts. Several variations are suggested including aspect level opinion mining (Cambria et al. 2013a), (Cambria et al. 2013b). This is also known as feature or attribute level opinion mining. Aspect 10

level opinion mining aims to discover aspects or entities of opinion mining and the corresponding sentiment orientation. Thus, it is split into two sub-tasks: opinion target extraction and sentiment classification. The challenges are the complicated expressions of opinions and the lack of annotated corpora at the fine-grained level. Recent techniques for fine-grained level opinion mining are among others: 

Unsupervised approaches Association mining algorithm (Hu & Liu 2004) for aspect detection and linguistic knowledge, such as meronymy relations (Popescu 2005) and part-whole patterns (Zhang et al. 2010) are considered. For aspects extraction Qiu et al. (Qiu et al. 2009) propose a “double propagation algorithm”. Additionally, rule-based methods are also suitable for detecting entities and aspects (Gindl, Weichselbraun & Scharl 2013).



Probabilistic generative model based approaches For aspects detection (Brody & Elhadad 2010) and sentiment detection (Lazaridou, Titov & Sporleder 2013) LDA topic models are adopted.

In different domains sentiments are expressed in a different way, i.e., the allocation of words is diverse in different domains (“long” in “the reaction time of the graphical card of the tablet is very long” is negative and in “the tablet’s warranty is very long” positive). Hereby, the question is how to take advantage of information from a domain provided with richly annotated corpora, for opinion mining to use other domains with little annotated corpora. Manual annotation costs a lot, which is rarely practical. This phenomenon is called Cross-domain opinion mining. So-called domain adaptation methods are supposed are the commonly used techniques for this opinion mining tasks. Recent techniques for cross-domain opinion mining are among others: 

Domain adaption based approaches In domain adaption methods, words that come from different domains are collated according to sentiment orientations (Blitzer, Dredze & Pereira 2007). An active learning method is given by (Li et al. 2013) to reduce disagreed labels in samples from different domains.



Cross-domain lexicon based approaches In these approaches, the original sentiment lexicons are adapted to the specific domain in such a way that they are suitable for target domains (Bollegala, Weir & Carroll 2011). 11

Most of the available sentiment resources (i.e., sentiment lexicons and annotated corpora) are in English. Thus, an analysis for texts in other languages, such as German, are difficult to execute. It is too expensive to label reliable corpora in a manual way as well as to create sentiment lexicons in another language. In the case of Cross-lingual opinion mining, the resources in the source language should help to identify the sentiment orientation in the target language. The already mentioned domain adaption methods and so-called transfer learning are used. Recent techniques for Cross-lingual opinion mining are among others: 

Combination of single lingual approaches Bilingual dictionaries and parallel corpora are utilized to create new annotated corpora (Mihalcea, Banea & Wiebe 2007). Machine translation techniques also reach this kind of annotated corpora, but are less restricted (Lambert 2015).



Cross-lingual lexicon based approaches In approaches, based on machine translation the coverage of vocabulary can be improved by a so-called label propagation algorithm (Gao et al. 2015).

2.4 Comparative Opinion Mining Comparing entities is a general way to express opinions. When reviewers make their opinion on a product, it seems self-evident to compare these with potential rival products in different aspects. The knowledge gained in comparative reviews can help “identify potential risks and further improve products or marketing strategies” (Xu et al. 2011). A comparative opinion is defined as a relationship of similarities or differences between two entities. Comparative opinion mining takes these entities as well as preferences of opinion holders into account. From comparative sentences, compared entities, comparative words and aspects can be extracted. For instance, given the sentence “Tablet X’s screen is better than tablet Y.”, “tablet X” and “tablet Y” are the compared entities, “better” is the comparative word and “screen” is the compared aspect. Because the word “better” expresses the preference clearly, “tablet X” is preferred. Comparative words or superlatives such as, e.g., “better”, reflect the preferences expressed in sentences. However, many comparative words, e.g., “longer”, express different positive or negative sentiment orientations in different contexts (cf. section 2.3). A rule-based method for this kind of sentence decomposes this problem into two sub-tasks (Jindal & Liu 2006): comparative sentence identification and comparative relation extraction. 12

Class Sequential Rules (CSRs) with class labels (i.e., “comparative” or “noncomparative”) and Label Sequential Rules (LSRs) applied on comparative sentences help solve these tasks, respectively. Another method divides comparative sentences into two categories: opinionated comparatives and comparatives with context-dependent opinions (Ganapathibhotla & Liu 2008). In the first case, comparative words are used. In the second, external information is needed. Pros and Cons in reviews contain information about the preferences of the reviewers. A CRF-based model can extract comparative relationships between products from customer reviews (Xu et al. 2011). Generally, each comparative word represents a comparative relation, respectively. Sentences that contain more than one comparative words are difficult to analyze. The CRF-based model considers such interdependencies (Jindal & Liu 2006).

2.5 Deep Learning for Opinion Mining Deep learning is a class of methods of optimization of artificial neural networks, which have numerous intermediate layers between the input layer and the output layer, and therefore have a large internal structure. In extensions to the learning algorithms for network structures with very few or no interlayers, as in the case of the single-layered perceptron, the methods of deep learning also enable a stable learning success (Goodfellow, Bengio & Courville 2016). Deep learning has become popular in visioning, speech recognition and natural language processing. In this section, we briefly introduce deep learning algorithms for natural language processing. Recent studies suggest building vectors of text features opinion mining without performing feature engineering. The task of extracting opinion expression is then a tokenlevel sequential labeling task. Recurrent Neural Networks (RNNs) extend a conventional feedforward neural network, processing input sequences with variable length. With DRNNs, opinion expressions from sentences can be filtered (Irsoy & Cardie 2014). They exceed the CRFs in their performance. The method arranges Elman-type RNNs one above the other. In bidirectional RNNs, the output at any given time can depend not only on the previous element of a sequence, but also on the future element. For example, to guess a missing word, one can analyze both the left and the right context of the missing word (Mikolov et al. 2011). Syntactic parsing is important because it determines the meaning from linguistic expressions. For this, a Compositional Vector Grammar (CVG) combines Probabilistic

13

Context-Free Grammars (PCFGs) with a syntactically untied RNN. Here, syntactic-semantic, compositional vector representations are learned (Socher et al. 2013). In another model called Recursive Neural Tensor Network (RNTN) a phrase is represented through word vectors and a parsing tree. Then, the vectors are computed by the same tensor-based composition function (Socher et al. 2013). A so-called Long Short-Term Memory (LSTM) is specifically designed to model long-term dependencies in RNNs. LSTM has no fundamentally different architecture to that of RNNs, but it uses a different function to compute the hidden states. Sequential models like RNNs and LSTMs are also used as powerful approaches for semantic composition (Tai, Socher & Manning 2015). In the so-called Dynamic Convolutional Neural Networks (DCNNs) for semantically modeling of sentences the network uses so-called “dynamic k-max pooling, a global pooling operation over linear sequences” (Kalchbrenner, Grefenstette & Blunsom 2014). The network treats variable-length input sentences and induces a feature graph over the sentences. An interesting work in this field introduces a neural probabilistic language model. On basis of the word representations a continuous representation for words and a probability function for word sequences is learned (Bengio et al. 2003). In the Continuous Bag-of-Words (CBOW) model, the current word to be learned is predicted based on the embedding of its context words. In contrast, the skip-gram model can predict the surrounding words by the embedding of the current word into the context. These both models as well as the popular word2vec  toolkit6 are proposed by Google (Mikolov et al. 2013). GloVe (Global Vectors for Word Representation) is an unsupervised learning algorithm, that extracts vector representations of words (Pennington, Socher & Manning 2014).

3. Foundations of Decision Making 3.1 Preference and Utility Preferences are important in the study of decisions such as in mathematical economics, social choice theory and opinion mining. Preferences are usually “modelled as a binary relation over the set of choices” (Brams & Fishburn 2007), (Shoham & Leyton-Brown 2008), (Kaci 2011). A set of choices (also outcome, alternative) for an agent which has the preference relation ≻ are named 6

and



is read “

is at least as good as

” where

https://code.google.com/p/word2vec/

14

,

∈ . Furthermore, it is expected, that ≽ is a complete and transitive relation. There are

two preference relations for ≽: for any , ’ ∈ , ≻ ’ iff7 ≽ ’ and ′ ⋡ Strict preference This preference relation is read is better than ’. ∼ ’ iff ≽ ’ and ′ ≽ Indifference And this is read the agent is indifferent between and ’. A utility function

maps a choice to a real number representing the degree of request.

There may exists more than such a function. The representation theorems which ensures the existence of such functions, formally are defined as follows (Fishburn 1969): Given the choices , ′ ∈

a utility function, :

→ represents

≽ if ≽ ’ iff

_ are 

≻ values 





_ of 

_



≻ if ≻ ’ iff



∼ if ∼ ’ iff



_

For instance, if



20 and

_

5, this leads to the preference

since 5 < 20. This means, the choices  a  ,

single  _

attribute 

price, 

or 

_

equally 

 and  in 

set 

_

 

notation 

(Acar et al. 2017). 

Normally, decisions are more complex. Therefore, choices are formalized as values  or elements of attributes. For instance, if we will buy a car, not only the price will be of interest, but also its color, its engine performance and even more. As mentioned above, MAUT concerns with such decision problems (Russell & Norvig 2009). Formally, the set of attributes is denoted by

. Then,



refer to a specific attribute in

 where ∈ 1, . . . , | | . With

these preliminaries, we can formalize the set of choices made by the cartesian product over the set of attributes. This set of choices is denoted by Ω where Ω the utility function

,..., The size of the

,..., ≽

, ,...,

⊇ . Now,

∶ →   is the (multi-attribute) utility function

has been expanded:

which represents ≽ iff ∀



,..., iff

∈ , ,...,

,...,

is i.e., 2| | , the assumption that u is additive helps to significantly reduce

the computational complexity. A typical additive function is, for example ,...,

7



. . .

Additivity

„iff“ means „if and only if“

15

,...,

where

∈ . Now, we can formulate an optimization task, namely that a rational

agent should make the choice with the maximum utility: ∶ where

arg max

Optimal choice



matches to maximal elements in

with respect to the utility function

(and

therefore means w.r.t. the preference relation ≽). We consider the class of decision making problems in such a way that

is a finite set, so-

called discrete choice problems. Examples are the selection of appropriate medical treatment, that meets the criteria of a patient. 3.2 Description Logics The signatures of description logics (Baader et al. 2003) can be given as a triple ,

,

, where denotes the set of atomic concepts, denotes the set of role names, denotes the set of atomic individuals.

We postulate the unique name assumption. This means that different individuals must also have different names, or vice versa, that one and the same individual must also have the same name. We denote concepts or classes by  and

, roles by  and , and individuals as  and

. Concept descriptions are defined in a common way from and

  are concept descriptions. Further, ∃ .   and ∀ . ⊔

description. The top concept  is abbreviation for

as

,

exist  if



⊓ , and   and



if  

  is a concept

and the bottom concept  for

.

For the semantic we need an interpretation for the presented syntax. An interpretation is a pair



,∙

is a set that can’t be empty, and ∙ is a so-called

where the domain

interpretation function. This function maps to every concept name every role name  a binary relation



:

∃ . ∀ .

≔ ≔

∈ ∈

a set



and to

. The function also defines: \



















| exists , | for all ,

, ,

∈ ∈

and ∈ implies ∈

There exist other extensions that are defined in a similar way.

16

In DLs, we distingue between terminological knowledge (TBox) and assertional ⊑

knowledge (ABox). A TBox is a set of concept inclusions ≡

⊑ and ⊑ . An ABox is a set of both concept assertions

if



and ,

and a concept definition is



∈ ,

,

, as well as role assertions ∈

where ,

where





a model) of ⊑

( ⊨

a concept ⊨

 

and

. ∅ holds. It

A concept is called satisfiable if there is an interpretation  for which that is said that a concept





is satisfiable with respect to a TBox  iff there is an interpretation (or ∅ holds. An interpretation

such that ⊑ ) iff

is a subset of

with respect to a TBox

if



(i.e. ⊆

satisfies a concept inclusion

holds . A concept

holds for every model of

⊑ ). An interpretation  satisfies a TBox

is subsumed by ( ⊑

of

or

 iff  satisfies every concept inclusion in

. A coherent TBox  is spoken if all concepts in  are satisfiable. We say that an ABox entails an assertion α (and write

⊨ ), if every model of

also satisfies . An ABox

called consistent with a TBox  if there exists an interpretation  that satisfies then call the pair

≔〈 ,

〉 a knowledge base. Further,

is satisfiable if

 and

is . We

is consistent

w.r.t.

. In the remainder, we will use the so-called instance check. Thus, for a knowledge

base

 and an assertion , one can check whether

A concrete domain ∩

∅ and for each ∈

holds.

,

  is defined as a pair

pred( ) is the set of predicate names of



.

  is

the domain of

  and

. The following assumptions have been applied: with arity n  there  is 



. According to

(Baader et al. 2003), functional roles are denoted with lower case letters, for example with . In description logics with concrete domains,

 is

partitioned into a set of functional roles and

one of ordinary roles. A role   is functional  if for every necessary that



,

∈   and

,

∈   it is

. Functional roles are explained as partial functions from

to

. Functional and ordinary roles are expressed with the existential quantification as

well as the universal quantification. A concrete domain is closed under negation (and is denoted by ). For this reason, a logical formula can be calculated which are in the so-called negation normal form (NNF). A formula is in NNF when the negation operators are only used between atomic statements. 3.3 Weighted Description Logics We will introduce an ontological approach to decision making. This approach can be considered as a generic framework, the so-called DL decision base (Acar et al. 2017). We use

17

an a priori preference relation over attributes (called the ontological classes). Thereby, an a posteriori preference relation over choices (called ontological individuals) can be derived by the so-called qualifier (section 0). Formally, a priori utility function attributes) is defined ( :



over

(the set of

). Additionally, a utility function u defined over choices,

which uses logical entailment, extends the utility function U to the subset of attributes. The utility function u was used for i.

mathematical-technical reasons, because a choice was defined as an individual and a corresponding outcome as a set of concepts, and flexibility, which means that the aggregate can take various forms, e.g., max, mean, or a customized one.

ii.

Modeling attributes in decision making has two steps: Step 1: Each attribute has been modeled by a concept. Step 2: For every value of an attribute a new (sub)concept to the set of concepts has been introduced. For instance, if color is an attribute to be modeled, it is simply represented by the concept ∈

  (i.e.,  

). A color can be regarded as a value, as if it were a concept of its

own. If blue is a value of the attribute color, the attribute set the concept Blue, as a subconcept of

 is simply extended by adding

. Furthermore, for each further available value a

subconcept is added to the attribute set. Please note: ‐

an axiom has been introduced to guarantee the disjointness. (e.g.



this procedure results in a binary term vector for



).

, because an individual c  (as a

choice) is either a member of the concept X or not (formally,



, where

is

the corresponding knowledge base). Given a total preference relation (i.e., ≽ ) over an ordered set of not necessarily atomic attributes

,

each concept

and iff ∈

a





:

function for

,





 

that

). The function

. Therefore, on can say, that “

The  utility  of  a  concept 



  is  denoted  by 



represents

(i.e.,

asigns an a priori weight to

makes the description logic weighted”. . The following applies: The greater the

utility of an attribute the more the attribute is preferable. Furthermore, the attribute set X can  be divided into two subsets: ‐

desirable denotes the set of attributes with non-negative weights, denoted



undesirable 

, i.e.,  ∈

iff

0 and



with

, and ∩



18

The meaning of the two definitions above is that any attribute that is not in desirable) must lie in

(not

and is therefore undesirable. In addition, it should be noted that an

attribute with weight zero can be interpreted as desirable with no utility. As mentioned above, a choice is an individual ∈

 .

 denotes the finite set of choices. To

determine a preference relation (a posteriori) over  (i.e., ≽ ), which respects ≽ , a utility function



is introduced.

indicates the  utility  of  a  choice    relative to the

attribute set

. Also, a utility function

 over attributes as an aggregator is introduced. For

simplicity, the symbol ≽ is used for both choices and attributes whenever it is evident from the context. The -utility is a particular

and is defined as follows: ≔

| ∈

and



 

Note:  

is called the sigma utility of a choice over



iff

i.e.,



∈ .

triggers a preference relation

.

Each choice corresponds to a set of attributes, which is logically entailed e.g., ⊨

. Due to the criterion Additivity (see section 3.1), each selection

corresponds to a result. Putting things (DL, 

and ) together:

a generic UBox (called Utility Box) is defined as a  pair 



,

,  where    is  a 

utility function over   and   is the utility function over  .  

a decision base can be defined as a triple consistent knowledge base, choices, and

Hint: the subscript

,

is a TBox and

, ,

where

is an ABox,



≔〈 ,

〉 is a

is the set of

is an UBox.

can be dropped and  can be written instead. 

Note: 

The decision  base is a (formal, logical) model in a decision context, or an entire decision support system.



The finite set of choices  is understood as a set of individuals.



The utility box encodes user preferences and generates a respective utility function.



provides assertional information about the choices as well as terminological knowledge information about the agent ability to reason over choices (as in the case of -utility). 19



In this approach, we limit ourselves to desirable attributes

. This means that

provided everything else is the same, everything that belongs to a concept is as least as desirable as something that belongs to its superconcept (i.e., upper  class  tablet  is at least as desirable as a tablet). It can be concluded that the more specific the attributes fulfill a choice, the more utility they will have. Moreover, it is seen that two choices are of the same desirability, if they belong exactly to the same concept. Example: We want to buy a tablet computer. Two alternatives are considered; an Acer Switch Alpha 12 and an Apple iPad Pro which fits the original purpose. The buyer’s decision base (background knowledge ( ,

,

), choices  

, and attributes mentioned in

) is given above. The language uses discrete domains. The domain €











with



. The partition

⊂ ℚ ,

€ ≔

€,

€,

€,



€,



∅, and

of the domain

€,



with ≔ € and ≔ € such that

is used and

with

≔ €

is defined as €



,

,



€ ∪



€ | ∈





, ∈

. Further predicates are similarly defined. Note:

€ is closed under negation. This means that we can invert the predicates in an obvious ,

was like



\0

and



,





. Other partitions are defined as follows:

| ∈



| ∈

0 . The remaining predicate names and functional roles are

also defined: ⊆

,

⊆ ⊆



,

,















, ⊆



,

, € ≡

, ,

. ⊑

,

,

, 710 g ,







,



.

,

,





⊑ ⊑

,

, ⊑ ,

,





⊑ , ⊑

,

12 ⊑



,

,∃ ,



1 ⊆









,







.



,

,



,

,

⊑ , ∃







,

⊓ .

,



,













,

e

, 769 € ,

, 12 inch ,



12

, 12,9 inch , ,

, 1250 g ,

, 629 € , ,

,

20

, 50),

( ∃

.

, 30 ,

, 40 ,

, 60

,

Figure 1: The Opinion Mining Architecture OMA

Considering

the agent is more interested in a tablet with a keyboard than in an upper

class tablet or in an inexpensive tablet. The utility of 30

40

70 and

50

30

can be calculated by 60

140. Thus,



4. OMA – An Architecture for Opinion Mining The Opinion Mining Architecture (OMA) we propose heavily bases on the approaches of natural language processing and machine learning presented in Section 2 and on decision making with weighted description logics presented in section 3. The idea of a separate representation and processing of knowledge and evaluation comes from the text understanding system SYNDIKATE (Hahn & Schnattinger 1997). Similarly, the separation according to knowledge base and so-called qualifier in SYNDIKATE we separate OMA according to opinion mining (preprocessing and analyzing texts as well as filtering out opinions) and decision support (evaluating extracted opinions). The system architecture for OMA (Figure 1) we propose serves the generation of opinions from texts like news, employee and public participation, expressions of opinions, political conversations, etc. (see step 1 in Figure 1).

21

The representation of the underlying domain (TBox) as well as the opinions expressed as assertions (ABox) use a description logic model (see step 2 in Figure 1). The TBox contains concepts which represents artefacts like compliance, rule, judgment, idea, sentiment, opinion, etc. The ABox contains assertions. In terms of content, it consists of opinions that are extracted from the sources of text. Whenever an opinion is stored in the ABox, different types of machine learning and natural language processing models carry out an evaluation. (see step 3 in Figure 1).

Figure 2: Sentiment scores for Sparkasse, Volksbank, Deutsche Bank and Commerzbank from January 2017 to May 2017

These models are presented in a so-called MBox (stands for methodology box). The evaluation provides a ranking of the opinions according to their utility. These weighted opinions are stored in the UBox (see 3.3). It should be noted that not every opinion can be weighted and therefore does not appear in the UBox. In view of OMA architecture, we intend to build a model for opinion mining in various domains such as sentiment mining for the financial sector. The results of a first attempt to determine sentiments for Deutsche Bank, Commerzbank, Volksbank and Sparkasse during the introduction of account management fees in spring 2017 has shown that OMA can deliver conclusive results. Starting from measured sentiment score for each of these banks, the sentiment scores for those banks fell, which have announced the introduction of a fee for account management in April 2017. As you can see in Figure 2 sentiment scores for the Sparkasse and Volksbank ran relatively uniformly from January to March 2017. In April, the score declined due to the announcement of account management fees. One month later in 22

May, after first account fees were reported on the account statement, the score fell significantly. For Deutsche Bank and Commerzbank such behavior couldn’t be observed, since these banks charge account fees for a long time already. Interestingly, this result could have been achieved by the fact that a supervised learning method had to be used to improve the results of the score calculation in addition to the preprocessing techniques of NLP, such as stop word lists and tokenization. Therefore, we used a Naïve Bayes classifier at document level and trained him with several hundred tweets. To select the right tweets, we use a bag-of-word model with unigrams. As a technological platform, we used OpenNLP.

5. Conclusion We have presented a methodology for opinion mining together with decision making based on machine learning and natural language processing methods for emerging opinions. May the approaches of opinion mining depend on concrete domains, as argued above, the principles underlying the ordering of opinions are general due to the use of description logics. Nevertheless, as weighted assertions are ubiquitous, one may easily envisage assertions with other content, e.g. data from IoT devices that provide incorrect values due to electronic fluctuations. The extension of OMA to exactly data from IoT is also part for our project. From a formal perspective, we intend to integrate additional extended descriptions logics within our framework. We could, for example, imagine methods mentioned in section 2.3 like supervised approaches with address semantic features.

23

Acknowledgements We would like to thank our colleague Prof. Dr. Jürgen Schenk, Director Study Program Financial Services for cooperation in the project “Sentiment analysis in the financial sector.” Klemens and Heike are winners of the Dr. Karl Helmut Eberle Foundation's award on the study “Digitization and Knowledge Transformation.”

24

References Acar, E, Fink, M, Meilicke, C, Thome, C & Stuckenschmidt, H 2017, 'Multi-attribute Decision Making with Weighted Description Logics', IFCoLog: Journal of Logics and its Applications, 4. Archak, N, Ghose, A & Ipeirotis, P 2007, 'Show me the money! Deriving the Pricing Power of Product Features by Mining Consumer Reviews', KDD'07: Proceedings of the Thirteenth International Conference on Knowledge Discovery and Data Mining, ACM, New York, NY. Baader, F, McGuinness, D, Narci, D & Patel-Schneider, P 2003, The Description Logic Handbook: Theory, Imlementation, and Applications, Cambridge University Press. Bengio, Y, Ducharme, R, Vincent, P & Jauvin, C 2003, 'A Neural Probabilistic Language Model', Journal of Machine Learning Research 3, 2003, pp. 1137-1135. Bhatia, P, Ji, Y & Eisenstein, J 2015, 'Better Document-level Sentiment Analysis from RST Discourse Parsing', EMNLP'15: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics. Bird, S, Klein, E & Loper, E 2009, Natural Language Processing with Python, O'Reilly Media. Blitzer, J, Dredze, M & Pereira, F 2007, 'Biographies, Bollywood, Boom-boxes and Blenders: Domain Adaptation for Sentiment Classification', ACL'07: Proceedings of the Forty-fifth Annual Meeting of the Association of Computational Linguistics, Association of Computational Linguistics. Bollegala, D, Weir, D & Carroll, J 2011, 'Using Multiple Sources to Construct a Sentiment Sensitive Thesaurus for Cross-Domain Sentiment Classification', EMNLP'11: Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics. Brams, S & Fishburn, P 2007, Approval Voting, Springer, 2nd edition. Brody, S & Elhadad, N 2010, 'An Unsupervised Aspect-Sentiment Model for Online Reviews', HLT-NAACL'10: Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, ACL. Cambria, E, Schuller, B, Liu, B, Wang, H & Havasi, C 2013a, 'Statistical Approaches to Concept-Level Sentiment Analysis', IEEE Intelligent Systems 28, 2013a, pp. 6-9. Cambria, E, Schuller, B, Liu, B, Wang, H & Havasi, C 2013b, 'Knowledge-Based Approaches to Concept-Level Sentiment Analysis', IEEE Intelligent Systems 28, 2013b, pp. 12-14. 25

Carnie, A 2010, Constituent Structure, Second Edition edn, Oxford University Press. Chajewska, U, Koller, D & Parr, R 2000, 'Making Rational Decisions using Adaptive Utility Elicitation', AAAI'00: Proceedings of the Seventeenth National Conference on Artificial Intelligence, AAAI Press, Austin, Texas. Chambers, N, Bowen, V, Genco, E, Tiam, X, Young, E, Harihara, G & Yang, E 2015, 'Identifying Political Sentiment between Nation States with Social Media', EMNLP'15: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, ACL, Lisbon, Portugal. Chevaleyre, Y, Endriss, U & Lang, J 2006, 'Expressive Power of Weighted Propositional Formulas for Cardinal Preference Modeling', KR'06: Proceedings of the Tenth International Conference on Principles of Knwoledge Representation and Reasoning, AAAI Press. Choi, Y & Cardie, C 2008, 'Learning with Compositional Semantics as Structural Inference for Subsentential Sentiment Analysis', EMNLP'08: Proceedings of the Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics. Dictionary.com 2002, The American Heritage® Idioms Dictionary, viewed 1 July 2017, . Die

Oberbadische

2017,

Gefühle

messbar

machen,

viewed

25

August

2017,

. Fishburn, P 1969, Utility Theory for Decision Making, John Wiley & Sons, Inc., New York, London, Sydney, Toronto. Gamon, M 2004, 'Sentiment classification on customer feedback data: noisy data, large feature vectors, and the role of linguistic analysis', COLING'04: Proceedings of the Twentieth International Conference on Computational Linguistics, Association for Computational Linguistics. Ganapathibhotla, M & Liu, B 2008, 'Mining Opinions in Comparative Sentences', COLING'08:

Proceedings

of

the

Twenty-second

International

Conference

on

Computational Linguistics, Association for Computational Linguistics. Gao, D, Wei, F, Li, W, Liu, X & Zhou, M 2015, 'Cross-lingual Sentiment Lexicon Learning With Bilingual Word Graph Label Propagation', Computational Linguistics 41, 2015, pp. 21-40.

26

Giang, P & Shenoy, P 2003, 'Two Axiomatic Approaches to Decision Making using Possibility Theory', European Journal of Operation Research, pp. 450-467. Gindl, S, Weichselbraun, A & Scharl, A 2013, 'Rule-based Opinion Target and Aspect Extraction to Acquire Affective Knowledge', WWW'13: Proceedings of the Twenty-second International Conference on World Wide Web, ACM, New York, NW. Goodfellow, I, Bengio, Y & Courville, A 2016, Deep Learning, The MIT Press. Hahn, U & Schnattinger, K 1997, 'Deep Knowledge Discovery from Natural Language Texts', KDD'97 - Proceedings of the 3rd International Conference on Knowledge Discovery and Data Mining, AAAI Press, Newport Beach, CA. Hu, M & Liu, B 2004, 'Mining and Summarizing Customer Reviews', KDD'04: Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, New York, NY. Irsoy, O & Cardie, C 2014, 'Opinion Mining with Deep Recurrent Neural Networks', EMNLP'14: Proceedings of the Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics. Jindal, N & Liu, B 2006, 'Mining Comparative Sentences and Relations', AAAI'06: Proceedings of the Twenty-first National Conference on Artificial Intelligence, AAAI Press. Joshi, M & Penstein-Rosé, C 2009, 'Generalizing Dependency Features for Opinion Mining', ACL'09: Proceedings of the Fourty-seventh Annual Meeting of the Association for Computational Linguistics and the Fourth International Joint Conference on Natural Language Processing, Association for Computational Linguistics. Kaci, S 2011, Working with Preferences. Less is More, Springer, Heidelberg, Dordrecht, London, New York. Kahraman, C 2008, Fuzzy Multi-Criteria Decision-Making. Theory and Applications, Springer, New York. Kalchbrenner, N, Grefenstette, E & Blunsom, P 2014, 'A Convolutional Neural Network for Modelling Sentences', ACL'14: Proceedings of the Fifty-second Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics. Keeney, R & Raiffa, H 1993, Decisions with Multiple Objectives: Preferences and Value Tradeoffs, Cambridge University Press, First published in 1976 by John Wiley & Sons, Inc. Kennedy, A & Inkpen, D 2006, 'Sentiment Classification of Movie Reviews Using Contextual Valence Shifters', Computational Intelligence 22, 2006, pp. 110-125.

27

Kim, S-M & Hovy, 2004, 'Determining the Sentiment of Opinions', COLING'04: Proceedings of the Twentieth International Conference on Computational Linguistics, Association for Computational Linguistics. Lafage, C & Lang, J 2000, 'Logical Representation of Preferences for Group Decision Making', KR'00: Proceedings of the Seventh International Conference on Principles of Knowledge Representation and Reasoning, Morgan Kaufmann Publishers, San Francisco, CA. Lafferty, J, McCallum, A & Pereira, F 2001, 'Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data', ICML'01: Proceedings of the Eighteenth International Conference on Machine Learning, Morgan Kaufmann Publishers. Lambert, P 2015, 'Aspect-Level Cross-lingual Sentiment Classification with Constrained SMT', ACL-IJCNLP'15: Proceedings of the Fifty-third Annual Meeting of the Association for Computational Linguistics and the Seventh International Joint Conference of the Asian Federation on Natural Language Processing, Association for Computational Linguistics. Lazaridou, A, Titov, I & Sporleder, C 2013, 'A Bayesian Model for Joint Unsupervised Induction of Sentiment, Aspect and Discourse Representations', ACL'13: Proceedings of the Fifty-first Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics. Lin, C & He, Y 2009, 'Joint Sentiment/Topic Model for Sentiment Analysis', CIKM'09: Proceedings of the Eighteenth ACM Conference on Information and Knowledge Management, ACM, New York, NY. Liu, B 2012, Sentiment Analysis and Opinion Mining, Morgan & Claypool Publishers. Li, S, Xue, Y, Wang, Z & Zhou, G 2013, 'Active Learning for Cross- omain Sentiment Classification', IJCAI'13: Proceedings of the Twenty-third International Joint Conference on Artificial Intelligence, AAAI Press. Manning, C, Surdeanu, M, Bauer, J, Finkel, J, Bethard, S & McClosky, D 2014, 'The Stanford CoreNLP Natural Language Processing Toolkit', ACL'14: Proceedings of Fifty-second Annual Meeting of the Association for Computational Linguistics: System Demonstrations, Association for Computational Linguistics. Mihalcea, R, Banea, C & Wiebe, J 2007, 'Learning Multilingual Subjective Language via Cross-Lingual Projections', ACL'07: Proceedings of the Forty-fifth Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics.

28

Mikolov, T, Kombrink, S, Burget, L, Cernocký, J & Khudanpur, S 2011, 'Extensions of Recurrent Neural Networks Langauge Model', ICASSP'11: Proceedings of the 2011 IEEE International Conference on Acoustics, Speech and Signal Processing, IEEE. Mikolov, T, Sutskever, I, Chen, K, Corrado, G & Dean, J 2013, 'Distributed Representations ofWords and Phrases and their Compositionality', Advances in Neural Information Processing Systems 26, 2013, pp. 3111–3119. Nopp, C & Hanbury, A 2015, 'Detecting Risks in the Banking System by Sentiment Analysis', Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, ACL, Lisbon, Portugal. O'Connor, B, Balasubramanyan, R, Routledge, B & Smith, N 2010, 'From Tweets to Polls: Linking Text Sentiment to Public Opinion Time Series', ICWSM'10: Proceedings of the Fourth International AAAI Conference on Weblogs and Social Media, AAAI Press. Paltoglou, G & Thelwall, M 2010, 'A Study of Information Retrieval Weighting Schemes for Sentiment Analysis', ACL'10: Proceedings of the Fourty-eighth Annual Meeting of the Association for Computational Linguistics, Association for Computer Linguistics. Pang, B & Lee, L 2004, 'A Sentimental Education: Sentiment Analysis Using Subjectivity Summarization Based on Minimum Cuts', ACL'04: Proceedings of the forty-second Annual Meeting of Association for Computational Linguistics, Association for Computational Linguistics. Pang, B & Lee, L 2008, 'Opinion Mining and Sentiment Analysis', Foundations and Trends in Information Retrieval 2, 2008, pp. 1-135. Pang, B, Lee, L & Vaithyanathan, S 2002, 'Thumbs up? Sentiment Classification using Machine Learning Techniques', EMNLP'02: Proceedings of the Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics. Pennington, J, Socher, R & Manning, C 2014, 'GloVe: Global Vectors forWord Representation', EMNLP'14: Proceedings of the 2014 Empirical Methods in Natural Language Processing, Association for Computational Linguistics. Popescu, A-M 2005, 'Extracting Product Features and Opinions from Reviews', HLT '05: Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing, Association for Computational Linguistics. Qiu, G, Liu, B, Bu, J & Chen, C 2009, 'Expanding Domain Sentiment Lexicon through Double Propagation', IJCAI'09: Proceedings of the Twenty-first International Joint Conference on Artificial Intelligence, AAAI Press. Reese, R 2015, Natural Language Processing with Java, Packt Publishing Ltd.

29

Řehůřek, R & Sojka, P 2010, 'Software Framework for Topic Modelling with Large Corpora', LREC'10: Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks, ELRA. Russell, S & Norvig, P 2009, Artificial Intelligence: A Modern Approach, Pearson Education, 3rd edition. Schnattinger, K & Hahn, U 1998, 'Quality-Based Learning', ECAI'98: Proceedings of the Thirteenth Biennial European Conference on Artificial Intelligence, John Wiley & Sons, Ltd. Shoham, Y 1997, 'Conditional Utility, Utility Independence, and Utility Networks', UAI'97: Proceedings of the Thirteenth Conference on Uncertainty in Artificial Intelligence, Morgan Kaufmann Publichers, San Francisco, CA. Shoham, Y & Leyton-Brown, K 2008, Multiagent Systems. Algorithmic, Game-Theoretic, and Logical Foundations, Cambridge University Press, New York. Socher, R, Bauer, J, Manning, C & Hg, A 2013, 'Parsing with Compositional Vector Grammars', ACL'13: Proceedings of the Fifty-first Annual Meeting on Association for Computational Linguistics, Association for Computational Linguistics. Socher, R, Perelygin, A, Wu, J, Chuang, J, Manning, C, Ng, A & Potts, C 2013, 'Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank', EMNLP'13: Proceedings of the Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics. Somasundaran, S, Namata, G, Wiebe, J & Getoor, L 2009, 'Supervised and Unsupervised Methods in Employing Discourse Relations for Improving Opinion Polarity Classification', EMNLP'09: Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics. Sun, S, Luo, C & Chen, J 2017, 'A Review of Natural Language Processing Techniques for Opinion Mining Systems, Volume 36', Information Fusion, pp. 10-25. Sutton, C & McCallum, A 2011, 'An Introduction to Conditional Random Fields', Foundations and Trends in Machine Learning 4, 2011, pp. 267-373. Taboada, M, Brooke, J, Tofiloski, M, Voll, K & Stede, M 2011, 'Lexicon-Based Methods for Sentiment Analysis', Computational Linguistics 37, 2011, pp. 267-307. Tai, K, Socher, R & Manning, C 2015, 'Improved Semantic Representations From TreeStructured Long Short-Term Memory Networks', ACL-ICNLP'15: Proceedings of the Fifty-third Annual Meeting of the Association for Computational Linguistics and the

30

Seventh International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, Association for Computational Linguistics. Tang, D, Qin, B, Wei, F, Dong, L, Liu, T & Zhou, M 2015, 'A Joint Segmentation and Classification Framework for Sentence Level Sentiment Classification', IEEE/ACM Transactions on Audio, Speech, and Language Processing 23, 2015, pp. 1750–1761. Turney, P 2002, 'Thumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of Reviews', Proceedings of the Fortieth Annual Meeting on Association for Computational Linguistics, Association for Computational Linguistics. Wang, S & Manning, C 2012, 'Baselines and Bigrams: Simple, Good Sentiment and Topic Classification', ACL'12: Proceedings of the fiftieth Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics. Xu, K, Liao, S, Li, J & Song, Y 2011, 'Mining comparative opinions from customer reviews for Competitive Intelligence', Decision Support Systems 50, 2011, pp. 743-754. Yang, B & Cardie, C 2015, 'Context-aware Learning for Sentence-level Sentiment Analysis with Posterior Regularization', ACL'14: Proceedings of the Fifty-second Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics. Yu, H & Hatzivassiloglou, V 2003, 'Towards Answering Opinion Questions: Separating Facts from Opinions and Identifying the Polarity of Opinion Sentences', EMNLP'03: Proceedings of the 2003 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics. Zhang, L, Liu, B, Lim, S & O'Brien-Strain, E 2010, 'Extracting and Ranking Product Features in Opinion Documents', COLING'10: Proceedings of the Twenty-third International Conference on Computational Linguistics, Association for Computational Linguistics. Zhang, W & Skiena, S 2010, 'Trading Strategies to Exploit Blog and News Sentiment', ICWSM'10: Proceedings of the Fourth International AAAI Conference on Weblogs and Social Media, AAAI Press.

31

The authors Prof. Dr. Klemens Schnattinger studied Business Informatics at the University of Mannheim with emphasis on Artificial Intelligence and Theoretical Computer Science and graduated in 1991. He earned his doctorate in computer science (Dr. rer. nat.) from the University of Freiburg in 1998 with Prof. Dr. Nebel, Artificial Intelligence, and Prof. Dr. Hahn, Computational Linguistics. He then became managing partner of a software company. Since 2004 he has been Professor of Computer Science and from 2007 to 2017 he headed the Center for IT Management and Computer Science at the Baden-Wuerttemberg Cooperative State University. Currently he teaches Mathematics, Logic, Artificial Intelligence, and researches in the areas of applied Machine Learning, Natural Language Processing and Logic.

Prof. Dr. Heike Walterscheid studied Business Administration and Economics at the University of Bayreuth. Currently, she teaches Economics at the Baden-Wuerttemberg Cooperate State University (DHBW), Germany. Walterscheid has been Assistant Professor of Economics at the Technical University of Ilmenau, Germany. She coordinates Academic Affairs at DHBW Loerrach, u.o. member of the board (Ausschuss) of Economic Systems and Institutional Economics of the “Verein für Socialpolitik” and ASIR, affiliated with the German “Gesellschaft fuer Informatik”. She directs “allpinion”, a private loose network, which is concerned with direct participation in social systems 4.0. Her main research areas and most of her publications are in the fields of Institutional and Political Economics, Social System Theory, Media and Digital Economics.

32

STARKE PARTNERSCHAFT Rund 750 Unternehmen arbeiten mit der DHBW Lörrach zusammen.

BEWÄHRTES STUDIENKONZEPT Verbindung von erstklassiger Lehrqualität mit maximalem Praxisbezug.

DIE BESTEN KÖPFE SICHERN Studienplätze werden passgenau durch Duale Partner mit den geeignetsten Kandidaten besetzt.

HOCHQUALIFIZIERTE NACHWUCHSKRÄFTE GESUCHT? Nehmen Sie die Sache selbst in die Hand! – Als Dualer Partner.  WWW.DHBW-LOERRACH.DE/AUSBILDUNGSPARTNER