Bayesian Techniques to Behavior Analysis

Bayesian Techniques to Behavior Analysis
Dr.NaveenBansal Profile Pic
Published Date:25-10-2017
Your Website URL(Optional)
Application of Bayesian Techniques to Behavior Analysis in Maritime Environments 1 1 2 Francesco Castaldo , Francesco A.N. Palmieri , and Carlo Regazzoni 1 Seconda Universit` a di Napoli (SUN), DIII, via Roma, 29 - 81031 Aversa (CE), Italy 2 Universit` a di Genova, DITEN, Via all’Opera Pia, 11 - 16145 Genova (GE), Italy Abstract. The analysis of vesselbehaviors andship-to-ship interactions in port areas is addressed in this paper by means of the probabilistic tool of Dynamic Bayesian Networks (DBNs). The dimensional reduc- tion of the state space is pursued with Topology Representing Networks (TRNs), yielding the partitioning of the port area in zones of different size and shape. In the training phase, the zone changes of interacting moving vessels trigger different events, the occurrence of which is stored in Event-based DBNs. The interactions are modeled as deviation from the common behavior prescribed by a single-ship normality model, in order to reduce the number of conditional probabilities to calculate and store in the DBNs. Inference on the networks is then carried on to ana- lyze the behavior of various ships and vessels maneuvering in the harbor. The results of the algorithm are showed by using simulated data relative toarealport. Keywords: Interaction Analysis, Ship-to-Ship Interactions, Dynamic Bayesian Networks, Topology Representing Network. 1 Introduction The sadly famous Costa Concordia accident 1, as other dramatic crashes hap- pened in recent years in port areas or near the coastlines 2, confirm that the design of monitoring systems able to supervise complex and crowded areas as harbors, coastlines, airports, etc., is very far from being considered a closed is- sue. Nowadays, these areas are monitored by a great number of high-quality sensors, but the lack of robust methodologies able to combine these volumes of data hinders to analyze and comprehend what is really happening in the area under surveillance. In this paper, we analyze vessels of different kind during the time they reside in generic port areas.The security of maritime environments may be jeopardized by a great number of different threats: ships moving too rapidly or too slowly, pairsofshipssailingtooclosetoeachother,smallvesselsobstructingthe passage for larger ships, and so forth. By understanding and labeling the movements in the area we could build an intelligent system, capable of providing alerts or warnings to the human operators (whose presence is obligatory in ports) when the detected situations are not acknowledged as normal. The issue is that in  c Springer International Publishing Switzerland 2015 175 S. Bassis et al. (eds.), Recent Advances of Neural Networks Models and Applications, Smart Innovation, Systems and Technologies 37, DOI: 10.1007/978-3-319-18164-6_17176 F. Castaldo, F.A.N. Palmieri, and C. Regazzoni crowded harbors it is likely to find many types of ships (motorboats, tugboats, container ships, etc.) interacting in many different ways with the other moving objects in the scene. In general terms, an interaction in maritime environments 3 occurs when a ship comes too close to another ship, or too close to a river, or to a canal bank. We focus on ship-to-ship interactions 3 4 in ports, i.e. when the presence and the movements of one ship affect the behavior of another, and vice-versa. The ships type, the navigation rules 5 of the country in which the port resides, the wind conditions, are all parameters that concur to define a normal interaction between vessels. We assume the interaction to be over when the two ships reach their destinations (for entering ships) or leave the port area (for exiting ships). In literature the ship-to-ship interaction problem has been mostly approached by analyzing the hydrodynamic phenomena arising when two or more watercraft are slightly spaced from each another 6 4. Bayesian reasoning 7 8 9 has beenextensivelyusedtostudytheinteractionofobjectsfordifferentapplications and in different scenarios, but little has been done for the behavioral analysis of pairs of ships. The reduction of the state space, necessary to carry out the event-based approach described in the paper, is achieved by means of Topol- ogy Representing Networks (TRNs), among which we choose the Instantaneous Topological Map 10. The paper is structured as follows. In Section 2 we analyze the techniques to reduce the state space and partition the area in zones. Section 3 describes the probabilisticapproachbasedonthe eventdetection andidentification.In Section 4 we drawn some results by using data generated in a simulated environment that replicates the port of Salerno, Italy. Section 5 is for conclusions and future developments. 2 Reduction of State Space with Topology Representing Networks In this paper the actors in play, namely ships and vessels of different kinds and shape, are treated as points moving in the 2D space described by the portion of sea included in the port. For the i-th ship we can define the state vector i i i T s =(x ,y ) , representing the position of the moving object at time t.The t t t analysis of behaviors and interactions between ships (and in general between moving objects) by evaluating the low-level state space trajectories as they are (without any modification) turns out to be quite a challenge, given the great variability of the state space vectors relative to multiple ships in port areas (even in small ones). However, we can exploit the fact that the “features” of a port (i.e. the position and the shape of the docks, the common routes of the ships, etc.) can be easily known a priori and do not change very often. If we are able to construct a topological map of the harbor, it is possible to design a higher-level algorithm where behaviors and interactions are emphasized and emerge with more clarity. The simplest way to build a map is to partition the area in zones of equal size with a rectangular grid. However, this approach ignores that ships andApplication of Bayesian Techniques to Behavior Analysis 177 vessels take only certain routes to enter or exit the port. The latter information is precious because we expect an “intelligent” map to be more precise in the zones where many ships pass through, and coarser in places rarely touched by ships. The Topology Representing Networks (TRNs) are an important class of algorithms that exploits at best the positional information of the actors in play, by building a map from a dataset of moving objects exploring the scene. The most famous TRN algorithms are the Self Organizing Maps (SOM) 11, but in recent times other approaches have been proposed, as the Growing Neural Gas (GNGs) 12 or the Instantaneous Topological Maps (ITMs) 10. In this paper the topological maps are built by means of ITMs, and this is motivated by the fact that ITMs are quite good in handling strongly-correlated data, as the one provided by ships and vessels sailing in the port. We do not report the explanation of the algorithm, as it is a straightforward implementation of the procedure detailed in 10. In order to build the map, we need to set only two parameters, namely the resolution e and the smoothing parameter  . max itm Therefore we assume to have the map of the environment, i.e. to have a set of N nodes, each of which corresponds to a zone. A zone can be defined as the n portionofthespacewhosepointsarecloser(respecttoafixeddistancedefinition, as for example the Euclidean distance) to the generator node (the “center” of the zone). The Bayesian models defined in Section 3 are based on zones changes triggered by the moving vessels in the area. More in detail, when the i-th vessel (a,b) i moves from zone a to zone b,anevent  = l → l is triggered, where l and a b a t l are the labels identifying two neighboring zones and t ∈ N is the time at which b (a,b) i the event occurs. The events  can be seen as the outcomes of the discrete t i random variable E , that will be the state of the Bayesian networks. If the vessel t (a,a) i remains in the same zone a for a T time, a still event  = l → l is max a a t detected. 3 Bayesian Models By means of an Event-based Dynamic Bayesian Network (E-DBN) 9 7, we 1 define a normality model Θ , relative to a target i-th ship sailing in the port. In i the E-DBN we encode the probability of the event ,giventhepreviousevent t i  through the following conditional probability (CPD), i t−Δ t 1 i i Θ = p(  ). (1) i t t−Δ t Ifthetargetshipisaloneintheport(orveryfarfromotherships)anddoesnot behave accordingly to the normality model (zig-zag trajectories,vessels stopping in the middle of the port, etc.), we can infer that the behavior is abnormal and a warning can be send to the operator. Things are different when other vessels are nearby the target ship, because a deviation from the normality could be due to interactionsbetween the vessels(as for instance a tugboat towing a cargo ship, a motorboat overtaking a sailboat, etc.). For this reason it is useful to define the following parameters: (a) the178 F. Castaldo, F.A.N. Palmieri, and C. Regazzoni i Euclidean distances d (called influence distances) between the target i-th ship j and the other j-th ships which reside or maneuver in the port area; (b) an i influence threshold τ , that, compared with the d distances, permits to verify i j if the target i-thshipandthe other j-th ships are close enough to interact i (d τ ) or not. Actually, in the experimental part of this paper (Section 4) i j we will use pairs of ships that, for simplicity, interact with each other during all the time they maneuver in the harbor, but in general the influence distances are very important in a multi-target scenario, where we do not know a priori who interacts with whom. Given these distances, it is possible to see the interactions between ships as deviation from the normality described by the model defined in Equation (1). i More specifically, if the j-th ship is very close to the target ship (i.e. d τ ), i j the following interaction model can be defined j m i i Θ = p(  , ), (2) i j t t−Δ t−Δ t t j where m=2,..,M denotes the type of interaction and  is the event relative j t−Δ t j i to the j-th ship, withtΔ ≤ Δ . Equation (2) can be written as t t j j i i i i i i p( , , ) p(  , )p(  ) i j j i i t t t t−Δ t−Δ t−Δ t−Δ t−Δ j t t t i i t t p(  , )= = , (3) i j t t−Δ j j t t−Δ i i t p( , ) p(  ) i j j i t−Δ t−Δ t−Δ t−Δ t t t t i i where p(  ) is the conditional probability defining the normality model i t t−Δ t of Equation (1). In other words, we can define the interaction as deviation j 1 i i from the normal model Θ , by adding two CPDs, namely p(  , ) j i t t−Δ t−Δ t t j i and p(  ), and in this way we can reduce the number of CPDs to store i j t−Δ t−Δ t t anduse.Inthispaperwefocusonaverycommontypeofship-to-shipinter- action between two vessels, but the proposed approach can be extended to the (unlikely) case of three and more interacting ships by adding the correspondent events in the model defined in (2). For instance, in the case of three ships we j i i n may define the CPD p(  , , n), where n denotes the third ship i j t t−Δ t−Δ t t t−Δ t n i andtΔ ≤ Δ . t t The Bayesian networks just introduced can be used to infer the behavior of ships maneuvering in the port, but only after an initial training process, in which the conditional probabilities within the models are estimated and stored. The latter CPDs describe the probability of a cause-effect relation between the events of nearby vessels, and are calculated with a maximum likelihood training algorithm 7, equivalent to counting the number of occurrences of the outcomes of the CPDs in the dataset, normalized to the total number of occurrences. The training of the network is performed with different datasets, related to behavior and interaction models. We point out that in large port areas different normality models (e.g. relative to different docks of the port) could exist, and the same for the interaction models. In such cases the number of models couldApplication of Bayesian Techniques to Behavior Analysis 179 significantly grow, and this is the main reason we decided to model the interac- tions as deviation from the normal path prescribed by the normality model. m If we assume to have M models Θ , m=1,..,M, in order to calculate the m probability that a new target ship behaves accordingly with the Θ model, the following cumulative normalized measure is proposed   k −1 1 m m m α = α + Θ , (4) k k−1 k k where k ∈ N denotes the number of detected events for the target ship and with m 1 0α 1. The normality model Θ can be always evaluated with the data of k 2 M the target ship, while the interaction models Θ ,.., Θ are used only if at least another vessel is nearby the target ship (information provided by the evaluation i of the influence distances d ). Given the vessels trajectories, for each couple of j m m events we calculate α and compare each model Θ with a threshold τ .If n k m none of the models is compatible with the trajectory (i.e. the α values result above the threshold for each m-model), we can infer that the ship behavior is m abnormal. We point out that α is a function that takes into account the past k history along with the probability of the current events, and its trend can be analyzed in real time to infer the behavior of the ship during the time period it resides in the harbor. 4 Preliminary Results In the following we report results of behavior analysis of ships in the Port of Salerno, Italy. The data are provided by a realistic simulator of trajectories, which reproduces the real structure of the port and generates the movements of ships entering the port (for simplicity we assume only entering ships, but the same reasoning can be applied when we have at the same time exiting ships). Figure 1 left depicts an image of the harbor of Salerno, and indicates in black the dock on which we focus our behavioral analysis. Figure 1 right depicts a frame of the simulator. 1 As explained in Section 3, the first step is to build the normality model Θ . This is accomplished with N = 150noisy trajectoriesof vesselsheading to the itm dock. These trajectories are used at first to build the Instantaneous Topological Map defined in Section 2, with e =5and  =0.1, and then to store the max itm CPDs of Equation (1) for different consecutive events. In Figure 1 left the ITM is superimposed to the port image. Given the normality model, it is possible to construct ship-to-ship interaction m models Θ , m =2,..,M, that are allowed in the portion of the port under surveillance. For simplicity, we build a single interaction model, and show how interactions not compatible with that model are robustly recognized. Given the European maritime rules in harbors 13 14, we define an interaction model 2 Θ relative to a sailboat and a motorboat trying to enter the port area at the same time. The navigation rules prescribe that the ship with the highest level of maneuverability (in this case, the motorboat) stops its engine, lets the other180 F. Castaldo, F.A.N. Palmieri, and C. Regazzoni Fig.1. Left: Satellite photo of the port of Salerno. We focus our attention to the right dock (indicated with black lines) where small vessels as motorboats and sailboats are allowed to land or depart (the other two major docks on the left are only for container or cargo ships). On the same image the ITM on which the eventsare gathered is superimposed. The green lines connect the neighbor nodes, and for each node we define the correspondent zone as the locus of points that are closer to that node, with respect to the others. Right: A picture taken from the simulator used in this paper, that accurately reproduces the shape of the port and generates realistic trajectories of ships in the area. ship pass through the entrance and only after enters the port. We call this a motorboat-sailboat interaction, because the target ship is always the motorboat and the other ship is always the sailboat, and we generate the model by using N = 300 noisy trajectories extracted from the simulator. We remark again ms that other interactions (for instance two motorboats entering the port in the same moment, a tugboat towing a container ship, etc.) are possible and can be m easily built in different Θ models. We have chosen empirically the value of τ =0.4and T =3. i max Once the ITM is created and models are assembled, inference on the data can be carried on. More in detail, a high-quality system has to guarantee two features: (a) low false alarm rates; (b) robust recognition of uncommon and abnormal behaviors or interactions. In order to assess the first feature, in the first experiment we test the Bayesian models with N = 200 noisy trajectories t 1 of two nearby ships that act as motorboat and sailboat of the interaction model 2 Θ . The single trajectoriesofthese ships are comparedwith the normality model 1 Θ , while at the same time the data from the two vessels are combined and 2 compared with the Θ model. In Figure 2 we depict the trend over the events of the cumulative measure defined in Equation (4). The analysis of the figure permits to draw the following conclusions: (a) the two trajectories singularly are almost always recognized as belonging to the normality model (their trends in very few cases and for little time are below the recognition threshold τ ). This i is true because in the model there is no indication of the time spent duringApplication of Bayesian Techniques to Behavior Analysis 181 Fig.2. This figure depicts the cumulative trends over the trajectories of interacting motorboats and sailboats. The two plots on top are relative to the single trajectories 1 compared with the normality model Θ , while the bottom plots are for the interaction 2 model Θ . the transition between events, therefore the fact that the motorboat stops at the entrance is not captured by the normality model; (b) the interaction model recognizes in most cases the motorboat-sailboat coupled behavior. Of course this is true when the first ship is the motorboat and the other is the sailboat (bottom left of Figure 2), and not when the ship roles areswitched (bottom right of Figure 2). In the second experiment we assess the ability of the system to alert the op- erator of strange or dangerous behaviors. We generate N = 100 trajectories t 2 relative to an interaction named tugboat-cargo, representative of the situations in which a large cargo ship is towed in the port by a tugboat. This type of inter- action is not allowed in the dock we are monitoring, therefore it is a dangerous situation that should be recognized. Even if the two ships are not a cargo and a tugboat but two motorboats or sailboats traveling together, this can be con- sidered still a noteworthy situation because two different ships so close in the port area could collide and cause relevant damages to the harbor structures. In Figure 3 are depicted the results, that are quite good and can be interpreted as 1 follows: the two trajectory, taken singularly, are compatible with the Θ model, 2 but their interaction is not recognized by the Θ model, except for very few cases and only for a few number of events. Such situation (two ships behaving in a normal way singularly but not interacting in a known way) can be easily reported to the operator, that can decide to intervene or not.182 F. Castaldo, F.A.N. Palmieri, and C. Regazzoni Fig.3. In this figure the models are compared with the data of ships interacting ac- cording to the tugboat-cargo model, in which one ship (the tug) tows the other (the cargo) into the port. While singularly the two ships are behaving correctly, this inter- action is not allowed in the small dock we are observing, and the trends of the various cumulative measures permit to automatically evaluate such situation and to report it to the operator. 5Conclusion This paper has presented an application of Bayesian networks for behavioral analysis of multiple ships in port areas. The idea is to preserve the port safety by classifying the movements of the different actors in the scene. The analysis is complicated by the fact that multiple ships can interact in many ways, with a number of interaction models that could become very large: the idea pursued in this paper is to relate the interactions to normality models, i.e. by modeling the interaction as deviation from the normal path taken by a ship maneuvering without other vessels in the port area. In this way we construct interactions starting from the normality model, reducing in this way the probabilistic data we haveto gatheranduse for inference. The computational loadofthe algorithm is quite low, because after the training step the inference is carried on by simply updating the cumulative measure for the normality and interaction models. Otherinformationcanbegatheredfrommovingshipsandusedtoenhancethe probabilistic model. For instance, the travel time of the ships into the zones can be saved along with the zone changes, and this information could be precious to recognize abnormal behaviors strictly connected with the vessel speed (i.e. ships that are too fast or slow, that stop into the middle of the port, etc.). Another useful information could be the initial position of the ship entering in a zone (i.e. from which part of the zone the ships usually enter), that could be used to construct, within the zone, a low-level tracking model by which follow the ship.Application of Bayesian Techniques to Behavior Analysis 183 The latter information could be used to anticipate the behavioral analysis at the level of the tracker instead of waiting for consecutive events (that for large zones could be triggered after quite long times). References 1. Costa concordia: What happened, 2. Cargo ship crashes into port control tower in genoa killing three, may/08/italian-cargo-ship-crashes-genoa 3. Barrass, B.: Ship Design and Performance for Masters and Mates. Elsevier Science (2004), 4. Eloot, K., Vantorre, M.: Ship behaviour in shallow and confined water: an overview of hydrodynamic effects through efd. In: Assessment of Stability and Control Pre- diction Methods for NATO Air and Sea Vehicles. NATO. Research and Technology Organisation (RTO), p. 20 (2011) 5. U. N. E. C. for Europe. Working Party on Inland Water Transport. CEVNI:, ser. TRANS/SC. 3/115/Rev. 2. UN (2002), 6. The Second International Conference on Manoeuvring in Shallow & Confined Wa- ters: Ship to Ship Interaction (May 2011) 7. Barber, D.: Bayesian Reasoning and Machine Learning. Cambridge University Press (2012) 8. Koller, D., Friedman, N.: Probabilistic Graphical Models: Principles and Tech- niques. MIT Press (2009) 9. Murphy, K.: Dynamic bayesian networks: Representation, inference and learning. Ph.D. dissertation, UC Berkeley, Computer Science Division (July 2002) 10. Jockusch,J.,Ritter,H.:Aninstantaneoustopological mappingmodelfor correlated stimuli. In:International JointConference onNeuralNetworks, IJCNN1999, vol. 1, pp. 529–534 (1999) 11. Kohonen, T.: Self-organized Formation of Topologically Correct Feature Maps. In: Anderson, J.A., Rosenfeld, E. (eds.) Neurocomputing: Foundations of research, pp. 509–521. MIT Press, Cambridge (1988), 12. Fritzke, B.: A growing neural gas network learns topologies. In: Advances in Neural Information Processing Systems 7, pp. 625–632. MIT Press (1995) 13. I. M. Organization, International Convention for the Safety of Life at Sea: consol- idated text of the 1974 SOLAS Convention, the 1978 SOLAS Protocol, the 1981 and 1983 SOLAS Amendments, ser. IMO Publication. IMO (1986), 14. I. M. Organization, ISPS Code: International Ship and Port Facility Security Code and SOLAS Amendments 2002 Adopted 12 December 2002, ser. IMO publication. International Maritime Organization (2003), Water and Natural Gas Demand Forecasting by Using Heterogeneous Data: A Preliminary Study Marco Fagiani, Stefano Squartini, Leonardo Gabrielli, Susanna Spinsante, and Francesco Piazza Department of Information Engineering Universit` a Politecnica delle Marche, Ancona, Italy m.fagiani,s.squartini,l.gabrielli,s.spinsante, Abstract. In this paper a preliminary study concerning prediction of domestic consumptions of water and natural gas based on genetic pro- gramming (GP) and its combination with extended Kalman filter (EKF) is presented. The used database (AMPds) are composed of power, water, natural gas consumptions and temperatures. The study aims to investi- gate novel solutions and adopts state-of-the-art approaches to forecast resource demands using heterogeneous data of an household scenario. In order to have a better insight of the prediction performance and properly evaluate possible correlation between the various data types, the GP ap- proach has been applied varying the combination of input data, the time resolution, thenumberof previousdata used for theprediction (lags) and the maximum depth of the tree. The best performance for both water and natural gas prediction have been achieved using the results obtained by the GP model created for a time resolution of 24h, and using a set of input data composed of both water and natural gas consumptions. The results confirm the presence of a strong correlation between natural gas and water consumptions. Additionalexperimentshavebeen executed in order to evaluate the effect of the prediction performance using long period heterogeneous data, obtained from the U.S. Energy Information Administration (E.I.A.). Keywords: domestic consumption forecasting, heterogeneous data, computational intelligence, genetic programming. 1 Introduction Nowadays, unlike the electrical energy scenario, where several databases and computational intelligence approaches exist, water and natural gas fields, as highlighted in Fagiani et al. 5, are affected by a severe lack of databases and studies. The only publicly available databases have been presented in Makonin et al. 9 and in Nasseri et al. 10. Additional data are available on the U.S. 1 Energy Information Administration (E.I.A.) site . For both the Almanac of 1  c Springer International Publishing Switzerland 2015 185 S. Bassis et al. (eds.), Recent Advances of Neural Networks Models and Applications, Smart Innovation, Systems and Technologies 37, DOI: 10.1007/978-3-319-18164-6_18186 M. Fagiani et al. Minutely Power dataset (AMPds) 9 and the U.S. Natural Gas Consumption 2 the temperature data are also available on Climate and National Climatic Data 3 Center , respectively. Concerningtheloadforecastingtechniques,manyapproacheshavebeenevalu- atedforthewaterscenarioandonlyafewforthenaturalgascase.Acomparative study between grey forecast model and RBF neural network model for annual water demand prediction has been presented in Liu and Chang 8. In order to forecast the urban water consumption, the combination of quantum particle swarm optimization (QPSO) algorithm with RBF neural network has been in- troduced by Zhu and Xu 17. Nasseri et al. 10 have introduced the application of genetic programming to forecast the monthly water demand of Tehran, and have discussed its application in combination with the extended Kalman filter. Specifically, in the state of the art only Azari et al. 1, Tabesh and Dini 16, and Pang 11 have recently discussed the effects of the application of multiple data type, like weather information or temperature, on the forecasting perfor- mance. Unfortunately, due to the presence of non-standard evaluation criteria and since each new method has been tested with a different database 5, it is extremely difficult to perform a comparisonbetween different approaches.In ad- dition, none of them have approached the study of correlation effects between the usage of different resources: water, natural gas and power. Moving from such a state of the art analysis, the main issues characterizing the methodological approach followed in this preliminary study are concern the identification of the most relevant and publicly available databases for the water and gas case studies and their use for experimentation (data heterogeneity is one of the key feature), the development of suitable Computational Intelligence algorithms for load forecasting in different operative contexts and, finally, a performance evaluation in according to the most used criteria in the field and comparison with similar techniques. The authors are confident that the spread of innovative monitoring systems, whicharemoreandmoreoftenbasedonlow-powerwirelessdevices13–15,will ensure a facilitation for collecting and making publicly available large amount of data containing multi-utilities information. Therefore, the AMPds 9 rep- resents the most complete database presently available that fulfils the author purposes. It is composed of power, water and natural gas meter data of a single house, recorded at one minute intervals for an entire year. Additional new data will be added in April of each year, starting 2013. Although the dataset refers to measurement of a single house for just one year, the diversity of resource types and the large amount of data allows to lead a study about correlation aspects between various resources, as well as correlation between resources and meteorological conditions. Concerningthe waterand naturalgasconsumption forecast,in this paper pre- dictions based on genetic programming (GP) and extended Kalman filter (EKF) are presented. The study aims to investigate novel solutions to forecast domestic 2 3 Water and Natural Gas Demand Forecasting 187 demands using heterogeneousdata composed ofdifferent resourcetypes, in order to research mutual correlation effects. In the future, such information can be use to improve both prediction and leakage detection techniques. Up to the authors’ knowledge, there are no others studies that have used the publicly available AM- Pds dataset to execute experiments concerning the short term load forecasting of water and natural gas consumption in a domestic scenario. This is the paper outline. In Section 2 the GP and EKF-GP algorithms are briefly described. The experimental tests are described and related results are commented in Section 3, whereas in Section 3.1 results comparison and further tests on monthly forecasting are described. Section 4 concludes the paper. 2 Computational Intelligence Algorithms Genetic Programming The genetic programming is inspired by the evolution process and, as this, it is based on three main criteria 2: heredity, variability and fecundity. In this way, an individual of a population will be able to adapt to an environment. The algorithm consists of the following steps: 1. Initialization: the parameters are set and the first generation is randomly created. 2. Selection:thebestindividualisselectedusingthesumoftheabsoluteerrors (SAE) and it is opportunely used for the creation of the new generation. 3. Control: the second step is repeated for the new generation until either a stop condition or the maximum generation is reached. At the end, the algorithm returns the best individual found since the beginning of the simulation. In order to have a better insight of the prediction performance and prop- erly evaluate possible correlation between different input data, the authors have applied the algorithm to each possible combination of input data and forecast resolution. In each test the best GP model has been sought varying both the number of previous data used for the prediction (lags) and the maximum depth of the node. Extended Kalman Filter The extended Kalman filter model produces a suboptimal solution using a first order linear approximation of the filter proposed by Kalman 7. The algorithm consists ofsuccessiveuses of predict and update equations,for each input/output relation. The main steps of the algorithm are: – Initialization: the first estimate is inferred from the only available infor- mation of the first input/output relation.188 M. Fagiani et al. ˆ Z − + K X k k −1 H F z ˆ X k/k−1 Fig.1. Extended Kalman filter block diagram – Iteration: the information of the previous estimate are used to correct the current estimate, minimizing the prediction mean-square error. In Fig. 1 the block diagram of the extended Kalman filter is shown. Let F be the transition matrix and A the Jacobian matrix of partial derivatives of F,the predict equations at time k are defined as: ˆ ˆ X = F X , (1) k/k−1 k/k−1 k−1/k−1 T P = A P A +Q . (2) k/k−1 k−1 k−1/k−1 k−1 k−1 The update equations, assuming that H is the observation model and Z is k the measurement vector, are given as: T T −1 K = P H (H P H +R ) , (3) k k k k/k+1 k/k−1 k k ˆ ˆ ˆ X = X +K (Z −H X ) , (4) k/k k/k−1 k k k k/k−1 P = P −K H P . (5) k k k/k k/k−1 k/k−1 EKF-GP The combination of extended Kalman filter and genetic programming, as indicated by Nasseri et al. 10, consists of the following steps: 1. The GP algorithm is performed and the best model is selected. 2. The EKF is applied at the dataset with the following settings: – the selected GP model is used as transition matrix F; – the noise covariances Q and R are set in accordance with the NMSE k k of the GP model;Domestic Water and Natural Gas Demand Forecasting 189 – the observation model H and the state variable X are vectors of dimen- sion m×1, with m equal to the lags number. 3. The EKF predicted states are re-computed using the GP model. Therefore, the k-th EKF predicted states, whose number depends on the lags as- sumed, are used as input variablesfor the selected GP model. The k-th predicted value is given as output of the model equation. More precisely, keeping the same selected GP function, new values of input lags are estimated, through the EKF, in order to improve the predictions. The original lags, that are used to compute the k-th prediction, are assigned as val- ˆ ues of the predicted state estimate vector, X , and, setting it as input, the k/k−1 result achieved by the GP model is used as measurement vector, Z ,10.Fur- k thermore, assuming that F is the GP function, in compliance with the definition of transition matrix, entails that the matrix A is diagonal and it is composed of partial derivatives of the GP function, F, with respect to the m input variables (lags) x ,x ,...,x , as depicted below. 1 2 m ⎛ ⎞ ∂F 0 ··· 0 ∂x 1 ∂F ⎜ ⎟ 0 ··· 0 ∂x ⎜ 2 ⎟ A = (6) m,m ⎜ ⎟ . . . . . . . . ⎝ ⎠ . . . . ∂F 00 ··· ∂x m Inaddition,alsothecovarianceestimate,P,isdiagonalwithdimensionm×m, and its initial values are all set equal to the noise covariance, R . k The selected operators for the GP are: plus (+), minus (−), product (×), n division (÷), power (x ), sine (sin) and cosine (cos). The MATLAB toolboxes EKF/UKF and GPLAB 6,12 have been used to execute the simulations. 3 Experimental Results The experiments have been conducted using the AMPds database, presented by Makonin te al. 9 and described in Section 1. Six combination of input data, for both natural gas and water prediction, composed by merging the available input informationasreportedin Table 1, havebeen evaluated. The 70%ofeachsethas been randomly selected and used for the training process; the remaining data (30%) have been used for testing the model and generate the reported results. The forecasts for 1h,6h,12h and 24h have been evaluated. For each set, different models have been trained and tested for 5, 3 and 2 lags and for a maximumtreedepthof20,15and10.Thepopulationsize,themaximumnumber of generation, and the cross-over and mutation probability have been set, and left untouched for all the simulation, to 100, 1000 and 0.1, respectively.190 M. Fagiani et al. The results have been mainly evaluated in terms of normalized mean square 2 error (NMSE) and coefficient of determination (R ) 4, defined as: N  2 (˜ y −y ) i i NMSE = , (7) σ ·N y N  1 2 (y −y˜) i i 2 N R =1− . (8) N  1 2 (y −y¯) i N where y indicates the i-th observed value, y˜ the corresponding i-th forecast i i value, the average y¯ and the variance σ of the N observed values. Additional y evaluation criteria taken into account are the mean square error (MSE), the mean absolute percentage error (MAPE) and the relative root mean square error (RRMSE). Table 1. Summary of the best performance obtained for each time resolution. The “Input data” column indicates the different combinations of available input resources. 1h 6h 12h 24h 2 2 2 2 Input data NMSE R NMSE R NMSE R NMSE R Natural Gas Prediction G 0.85 0.15 0.37 0.62 0.31 0.68 0.25 0.75 GT 0.46 0.53 0.39 0.60 0.30 0.69 0.28 0.71 WG 0.84 0.16 0.39 0.60 0.33 0.67 0.24 0.76 WGT 0.79 0.20 0.39 0.61 0.29 0.71 0.28 0.71 WGE 0.88 0.11 0.40 0.60 0.31 0.69 0.27 0.72 WGET 0.86 0.14 0.39 0.60 0.33 0.67 0.28 0.72 Water Prediction W 0.94 0.06 0.42 0.57 0.45 0.55 0.43 0.56 WT 0.95 0.05 0.44 0.56 0.44 0.56 0.43 0.57 WG 0.92 0.08 0.45 0.55 0.48 0.51 0.41 0.58 WGT 0.92 0.08 0.46 0.54 0.53 0.47 0.45 0.54 WGE 0.90 0.10 0.45 0.55 0.54 0.46 0.44 0.55 WGET 0.91 0.09 0.46 0.53 0.50 0.50 0.43 0.57 W = water G = natural gas E = electric power T = temperature Thebestresultsobtainedforeachtimeresolutionandinputsetarereportedin Table1.Forthenaturalgas,Forthenaturalgas,decreasingthetimeresolutionof the predictionproducesaclearperformanceimprovement.The bestperformance hasbeenobtainedfora24htimeintervalusingasinput5lagsofbothnaturalgas and water consumption, and a tree depth of 15. For this condition the obtained MSE is0.099,theRRMSE is39.24%,andtheMAPE is25.23%.Onthecontrary, the water prediction performance show a good improvement decreasing the timeDomestic Water and Natural Gas Demand Forecasting 191 Table 2. Comparison of the best results obtained with the GP and EKF-GP for each time resolution. 1h 6h 12h 24h 2 2 2 2 NMSE R NMSE R NMSE R NMSE R Gas Prediction WGT G WGT WG GP 0.79 0.20 0.37 0.62 0.29 0.71 0.24 0.76 EKF-GP 0.79 0.20 − − 0.32 0.68 0.23 0.77 Water Prediction WGE W WT WG GP 0.90 0.10 0.42 0.57 0.44 0.56 0.41 0.58 EKF-GP 0.90 0.10 0.44 0.56 0.44 0.56 0.58 0.42 −: no results, derivatives differs too much. Table 3. Results comparison with the state-of-the-art studies. 2 Tech. MSE NMSE R Gas Prediction (WG) GP 0.007 0.412 0.584 Bakker et al. 3 Adaptative - - 0.658−0.802 Tabesh and Dini 16 Fuzzy 0.042 0.465 0.760 Tabesh and Dini 16 Neuro-fuzzy 0.007 0.064 0.936 resolution from 1 hour to 6 hours, and a smooth improvement from 6 to 24 hours. However, the best performance is reached for a 24h time interval using as input 2 lags of both water and natural gas consumption, and a tree depth of 20. For this condition the obtained MSE is 0.0070, the RRMSE is 27.37%, and the MAPE is 21.15%. In both natural gas and water prediction the correlation between the resources is evident. The results obtained applying the EKF-GP approach at the best performing GP model of each resolution time are reported in Table 2. The prediction perfor- mance remains almost the same introducing the EKF, except for the 24h water prediction, for which its performance decrease drastically. 3.1 Further Remarks In Table 3 a comparison with results achieved in the state-of-the-art experi- ments, concerning the 24h resolution water forecasting, is reported. The au- thors are aware that the results achieved by GP approach are lower than those of the comparable studies. However, it should be noted that, differently from the AMPds, which data have been collected in 1 year, the database used in Tabesh and Dini 16 is composed of about 4,300 samples, over 12 years of recording, with both temperature and humidity information. Similarly, Bakker et al. 3 performed the forecasting with a database of 210,336 samples collected over 6 years.192 M. Fagiani et al. Table 4. Comparison of the best results obtained for the GP approach and the results achieved applying the EKF-GP, for each data combination. GP EKF-GP 2 2 NMSE R NMSE R Maine 0.166 0.822 0.231 0.768 Maine + Temp. 0.103 0.896 0.032 0.968 Illinois 0.069 0.930 0.104 0.896 Illinois + Temp. 0.068 0.931 0.061 0.938 Louisiana 0.199 0.798 0.156 0.844 Louisiana + Temp. 0.153 0.845 0.153 0.845 Therefore, in order to have a better insights on the prediction capability of the GP and EKF-GP using heterogeneous data collected over long period, the approaches have been applied to forecast the monthly natural gas demands of three U.S. states, using the information available on U.S. Energy Information Administration (E.I.A.) site from 1989 to 2013, as described in Section 1. The monthly data of 3 states have been chosen: Maine, Illinois and Louisiana, representing a cold, a mild and a hot climate, respectively. For each state two different data sets have been evaluated: one composed of both gas consumptions and temperature data, and the other composed of gas consumption data only. As for the previous approach, 70% of each set has been randomly selected and used for training the corresponding GP model, the remaining data (30%) has been used for the testing and generatingthe reported results. The results ofboth GP and EKF-GP combination are shown Table 4. As seen for the domestic prediction, the introduction of heterogeneous data resultsinaperformanceimprovement,remarkableinthe caseofMainedata.The application of the EKF-GP approach generates an additional enhancement for the combined data. Moreover,although the obtained resultscannot be compared withstate-of-the-artstudies,theperformancereachedinTable4seemtobegood and representing a valuable starting point for future developments. 4Conclusion In this paper the authors present a preliminary study concerning the use of het- erogeneousdata to forecastthe domestic consumptions of natural gas and water. Unlike heterogeneous data applications that have been shown in the state-of-the art studies 1,11,16, which experiments have been executed using combinations of a single resource type with weather conditions or temperatures, the data het- erogeneity expressed in this paper is extended to the combination of multiple resource types. Using the AMPds 9 database the authors have performed, and up to their knowledge no other study has done it, forecasting experiments of water and natural gas consumption in a household scenario. The prediction experiments have been computed using the genetic program- ming paradigm, and its combination with the extended Kalman filter has beenDomestic Water and Natural Gas Demand Forecasting 193 also evaluated. In order to analyse the suitability of the GP and EKF-GP al- gorithms for prediction purposes, experiments have been conducted using the monthly gas consumption of U.S. States, in combination with temperature in- formation. In the domestic prediction experiments, the best performance for both water and natural gas prediction are achieved using only the GP model created for a time resolution of 24h, with a set of input data composed of both water and natural gas consumptions. This results point out the evident correlation between water and natural gas consumptions. This information leads to prove the useofnaturalgasforthe productionofhot waterforthe evaluated household 4 scenario. In confirmationof this, in the AMPds FAQ within the appliances that use natural gas is reported the instant hot water unit. The results obtained with the long period data have shown the effectiveness of the EKF-GP approach. The forecasting performance obtained with the GP model exhibited a general improvement applying the EKF method. As for the domestic forecasting, the combination of consumptions and temperatures has shown better results than the forecast computed using the consumptions data alone, denoting the advantage of heterogeneous data utilization as well as the strong correlation among them. For this reasons, the authors will go beyond the preliminary status of the present work and develop further computational intelligence and machine learn- ing techniques for comparativepurposes. Moreover,as soon as the new yeardata ofthe AMPds will be released,this is allowa properevaluation ofthe seasonality issue. References 1. Azari, A., Shariaty-Niassar, M., Alborzi, M.: Short-term and Medium-term Gas Demand Load Forecasting by Neural Networks. Iranian Journal of Chemistry and Chemical Engineering (4), 77–84 (2012) 2. Babovic, V., Abbott, M.B.: The Evolution of Equations from Hydraulic Data Part I: Theory. Journal of Hydraulic Research 35(3), 397–410 (1997) 3. Bakker, M., Vreeburg, J., van Schagen, K., Rietveld, L.: A Fully Adaptive Fore- casting Model for Short-term Drinking Water Demand. Environmental Modelling & Software 48, 141–151 (2013) 4. Bennett, N.D., Croke, B.F., Guariso, G., Guillaume, J.H., Hamilton, S.H., Jake- man, A.J., Marsili-Libelli, S., Newham, L.T., Norton, J.P., Perrin, C., Pierce, S.A., Robson, B., Seppelt, R., Voinov, A.A., Fath, B.D., Andreassian, V.: Characterising Performance of Environmental Models. Environmental Modelling & Software 40, 1–20 (2013) 5. Fagiani, M., Squartini, S., Gabrielli, L., Pizzichini, M., Spinsante, S.: Computa- tional Intelligence in Smart Water and Gas Grids: An Up-to-date Overview. In: 2014 International Joint Conference on Neural Networks (IJCNN), pp. 921–926 (July 2014) 4 M. Fagiani et al. 6. Hartikainen, J., Solin, A., Sa ¨rkk¨ a, S.: Optimal Filtering with Kalman Filters and Smoothers - A Manual for MATLAB Toolbox EKF/UKF Version 1.3. Department of Biomedical Engineering and Computational Science, Aalto University School of Science (2011), 7. Kalman, R.E.: A New Approach to Linear Filtering and Prediction Problems. Transactions of the ASME – Journal of Basic Engineering 82(Series D), 35–45 (1960) 8. Liu, J., Chang, M.: Application of the Grey Theory and the Neural Network in Water Demand Forecast. In: 2010 Sixth International Conference on Natural Com- putation (ICNC), vol. 2, pp. 1070–1073 (2010) 9. Makonin, S., Popowich, F., Bartram, L., Gill, B., Bajic, I.V.: AMPds: A Public Dataset for Load Disaggregation and Eco-Feedback Research. In: IEEE Electrical Power and Energy Conference, pp. 1–6 (2013) 10. Nasseri, M., Moeini, A., Tabesh, M.: Forecasting monthly urban water demand using extended kalman filter and genetic programming. Expert Systems with Ap- plications 8(6), 7387–7395 (2011) 11. Pang, B.: The Impact of Additional Weather Inputs on Gas Load Forecasting. Ph.D. thesis, Marquette University (2012) 12. Silva, S., Almeida, J.: Gplab - A Genetic Programming Toolbox for MATLAB. In: Proc. of the Nordic MATLAB Conference (NMC-2003), pp. 273–278 (2005) 13. Spinsante, S., Pizzichini, M., Mencarelli, M., Squartini, S., Gambi, E.: Evaluation of the Wireless M-Bus Standard for Future Smart Water Grids. In: 9th Interna- tional Wireless Communications andMobile ComputingConference, pp.1382–1387 (2013) 14. Spinsante, S., Pizzichini, M., Mencarelli, M., Squartini, S., Gambi, E., Piazza, F.: Wireless M-Bus Sensor Networks for Smart Water Grids: Analysis and Results. International Journal of Distributed Sensor Networks (2014) (to appear) 15. Squartini, S., Gabrielli, L., Mencarelli, M., Pizzichini, M., Spinsante, S., Piazza, F.: Wireless M-Bus Sensor Nodes in Smart Water Grids: The Energy Issue. In: FourthInternationalConference onIntelligentControl andInformationProcessing, pp. 614–619 (2013) 16. Tabesh, M., Dini, M.: Fuzzy and Neuro-fuzzy Models for Short-term Water De- mand Forecasting in Tehran. Iranian Journal of Science & Technology, Transaction B, Engineering 33(B1), 61–77 (2009) 17. Zhu, X., Xu, B.: Urban Water Consumption Forecast Based on QPSO-RBF Neural Network. In: Eighth International Conference on Computational Intelligence and Security, pp. 233–236 (2012)Radial Basis Function Interpolation for Referenceless Thermometry Enhancement 1 2 3 3,4 Luca Agnello , Carmelo Militello , Cesare Gagliardo , and Salvatore Vitabile 1 Department of Chemical, Management, Computer, and Mechanical Engineering, University of Palermo, Palermo, Italy 2 Institute for Molecular Bioimaging and Physiology, National Research Council (IBFM CNR-LATO), Cefalù, Italy 3 Department of Biopathology, Medical and Forensic Biotechnologies, University of Palermo, Palermo, Italy 4 MIRC srl, Academic spin-off of the University of Palermo, Catania, Italy,, fcesare.gagliardo, Abstract. MRgFUS (Magnetic Resonance guided Focused UltraSound) is a new and non-invasive technique to treat different diseases in the oncology field, which uses Focused Ultrasound (FUS) to induce necrosis in the lesion. Tem- perature change measurements during ultrasound thermo-therapies can be per- formed through magnetic resonance monitoring by using Proton Resonance Frequency (PRF) thermometry. It measures the phase variation resulting from the temperature-dependent changes in resonance frequency by subtracting one phase baseline image from actual phase. Referenceless thermometry aims to reduce artefacts caused by tissue motion and frequency drift, fitting the back- ground phase outside the heated region. The aim of this contribution is to pro- pose a novel background phase reconstruction method using Radial Basis Function (RBF) interpolation. The effectiveness of the method has been demonstrated by comparing it against the classical PRF shift and polynomial referenceless approach. The comparison evaluates temperature rises in uterine fibroids during MRgFUS treatments on a set of 10 patients. Keywords: Radial Basis Function, Interpolation, Referenceless Thermometry, Artificial Neural Network, MRgFUS. 1 Introduction Hyperthermia is a type of clinical treatment in which body tissues are exposed to high temperatures that can kill pathological lesion, like uterine fibroids 2. In MRgFUS treatments 78, high temperatures are applied on local and small areas by using ultrasound beams that deliver energy to heat the tumour. MRgFUS treatment is per- formed using the ExAblate 2100 equipment (InSightec, Haifa, Israel), integrated with a Signa HTxt MR scanner (GE Medical Systems, Milwaukee, WI). Thermal ablation of fibroids tissue is done using sonication process: the tissue is heated with Focused © Springer International Publishing Switzerland 2015 195 S. Bassis et al. (eds.), Recent Advances of Neural Networks Models and Applications, Smart Innovation, Systems and Technologies 37, DOI: 10.1007/978-3-319-18164-6_19

Advise: Why You Wasting Money in Costly SEO Tools, Use World's Best Free SEO Tool Ubersuggest.