Question? Leave a message!

What is Business Intelligence

What is Business Intelligence 10
Topics in Business Intelligence Lecture 1: Introduction to BI case study Tommi Tervonen Econometric Institute, Erasmus University RotterdamWhat is Business Intelligence (BI) BI refers to computerbased techniques used in spotting, diggingout, and analyzing business data, such as sales revenue by products and/or departments or associated costs and incomes. BI technologies provide historical, current, and predictive views of business operations. Business Intelligence often aims to support better business decisionmaking. of BIExamples of BIExamples of BIExamples of BIBI framework Watson Wixom, 2007Main components in BIKnowledge discovery processWhy data mining Tremendous amount of data Walmart Customer buying patterns a data warehouse 7.5 Terabytes large in 1995 VISA Detecting credit card interoperability issues 6800 payment transactions per second High dimensionality of data Many dimensions to be combined together High complexity of data Timeseries data, temporal data, sequence data Spatial, spatiotemporal, multimedia, text and Web dataData mining Subtypes: Text mining: mining of patterns from text Web mining: discovering patterns from the webData mining: predictive analysis types Classi cation of observations to (possibly ordered) classes, e.g. credit card transactions to normal or fraudulent ones. Prediction is similar, but instead of assignment to classes, we try to predict the value of a numerical variable, e.g. amount of credit card purchase. Association rules or anity analysis tells what is associated with the observations. Recommender systems (e.g. use association rules.Data mining: preanalyses Data visualization allows \easy" overview of the data. Data exploration often needs to be done with large data sets to answer more vague questions. Similar variables and observations can be aggregated to get a better picture of the data. Data reduction consolidates a large number of variables or cases into a smaller set. Correlation principal component analyses.What is 'data' Data can essentially be: 1 Continuous ordered values with a scale. E.g. client monthly spending (e), speed of car (km/h) 2 Categorical discrete, possibly ordered values. E.g. car class (small family car, large family car, executive, ...), bank customer credit class (A, B, C, D) Often data is categorical due to form of reporting (e.g. from questionnaires: monthly salary)Data mining methods for BI Mostly: Statistical methods for analysis of continuous variables Machine learning for analysis of categorical variables Variables are divided into predictors and responsesData nature methods Continuous Categorical No response response response Continuous Linear regression Logistic regression Principal components predictors Neural nets Neural nets Cluster analysis knearest neighbors Discriminant analysis knearest neighbors Categorical Linear regression Neural nets Association rules predictors Neural nets Classi cation trees Regression trees Logistic regression Naive Bayes Ordered categorical variables (e.g. 1, 2, 3) can often be converted to continuous ones Continuous variables can always be converted to categorical ones through frequency analysis (binning)Data mining processLearning modes In unsupervised learning, no outcome variable is predicted.Learning modes In supervised learning the model is trained to predict a known response. The data needs to be split into training and test sets.Supervised learning with linear regression x = 200, y =Data mining process 1 Develop an understanding of the purpose of the data mining project 2 Obtain the dataset to be used in the analysis 3 Explore, clean, and preprocess the data 4 Reduce the data, if necessary, and (in supervised learning) separate into training, test, and validation sets 5 Determine the data mining task (classi cation, prediction, etc) 6 Choose the technique to be used 7 Apply algorithms 8 Interpret results 9 Deploy modelQCourse organization Lectures: 1st Introduction to BI case study 2nd Data reduction 3rd Model validation 4th Student lecture: Naive Bayes and kNN, Classi cation trees 5th Student lecture: Logistic regression 6th Student lecture: Neural nets 7th Overview of results, comparison with (yet another) test set, feedbackCourse learning objectives 1 Knowledge of basic principles of data warehouses 2 Comprehension of business implications of BI and data mining 3 Application of a single data mining classi cation method 4 Evaluation of data mining resultsCourse evaluation material Evaluation: Student lecture case analysis (100) Student lectures have mandatory attendance (1 miss allowed) Online material (all will be available My slides from the rst 3 lectures Slides of the student lectures Scienti c papers Course book: Shmueli, Patel Bruce, \Data mining for Business Intelligence" helps in making the student lecture but is not mandatoryStudent lectures Prepared in pairs or small groups Each lecture should consist at least the following: 1 Theoretical explanation of the method 2 An application of the method to a simple case 3 Presentation of reallife BI applications of the method 4 Analysis of the case study with the method Each lecture should be 40mins + 5min discussion: expect to spend 2 weeks in preparationCase study Direct mailings to potential customers (\junk mail") can be an e ective way to market a product of service. However, most junk mail is of no interest to majority of people, and ends up being thrown away. More directed marketing to highly potential customers saves waste e ort, and consequently lowers costs and increases pro ts.Case study scope Our customer is a Dutch charity organization that wants to be able to classify it's supporters to donators and nondonators. The nondonators are sent a single marketing mail a year, whereas the donators receive multiple ones (up to 4). Tasks: 1 Develop a data mining model for classifying the customers to donators and nondonators 2 Explain through the model which factors are important in deciding who is a donatorCase study data Information about donators in 8 variables: TIMELR TIME since Last Response (nr weeks) TIMECL TIME as CLient (nr years) FRQRES FReQuency of RESponse (to mailings) MEDTOR MEDian of Time Of Response AVGDON AVeraGe DONation (per responded mailing) LSTDON LaST DONation ANNDON Average ANNual DONation DONIND Donation indicator in the considered mailing (response) Training and test sets of over 4000 customersTools Spreadsheet software (e.g. gnumeric, OpenOce calc, or Excel) RapidMiner: an opensource, cross platform tool with available commercial supportMotivation: current directions in BI Packaged analytic applications delivered as both on premises software and software as a service (SaaS) will push control of the information used for decision making toward business units and away from IT organizations. The economic crisis will reveal which enterprises have a sound information infrastructure and which do not. The application of social software to the collaborative decision making process will demonstrate the business value of the information coming from BI systems by directly tying it to decisions made. Gartner Inc., 2009Rhine's paradox Joseph Rhine was a parapsychologist in the 1950's who hypothesized that some people had ExtraSensory Perception. He devised an experiment where subjects were asked to guess 10 hidden cards red or blue. He discovered that almost 1 in 1000 had ESP they were able to get all 10 rightRhine's paradox He told these people they had ESP and called them in for another test of the same type. Alas, he discovered that almost all of them had lost their ESP. What did he concludeYou should't tell people they have ESP. It causes them to lose it.Bonferroni's principle \If you look for interesting patterns in more places than your amount of data will support, you are bound to nd crap"1st week of case study (Download, install, and explore RapidMiner) 1 Develop an understanding of the purpose of the data mining project 2 Obtain the dataset to be used in the analysis 3 Explore the data (Import data into RapidMiner)