New SpikeOMatic Tutorial

New SpikeOMatic Tutorial Christophe Pouzat Copyright C Pouzat 2006 Contents 1 Introduction 4 1.1 Never Used SpikeOMatic before? . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.2 About this New Release . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.3 General Remarks on the Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.4 A Sketch of the Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.5 Your Job . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.6 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.7 Required R Packages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.8 License . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.9 Using this tutorial . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 1.10 R version used . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 1.11 Setting some variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2 Starting Up 8 2.1 Generating and Loading Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2.2 Loading Data and Creating rawData Object . . . . . . . . . . . . . . . . . . . . . 9 2.2.1 rawData Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.2.2 Creating Our First rawData Object . . . . . . . . . . . . . . . . . . . . . . 9 2.3 Getting Info and“Interacting”with rawData Object . . . . . . . . . . . . . . . . . 12 2.4 See What’s in a rawData Object . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2.4.1 Show Method for rawData Objects . . . . . . . . . . . . . . . . . . . . . . 12 2.4.2 Plot Method for rawData Objects . . . . . . . . . . . . . . . . . . . . . . . 13 2.4.3 Getting an Overall View on rawData Objects . . . . . . . . . . . . . . . . . 15 2.4.4 summary Method for rawData Objects . . . . . . . . . . . . . . . . . . . . 16 2.4.5 Exploiting the“Matrix”Features of rawData Objects . . . . . . . . . . . . . 18 3 Detecting Spikes 19 3.1 Filtering rawData Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.2 Detecting Spikes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 3.2.1 The markedPP Class . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 3.2.2 Checking Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 3.2.3 Detecting both Peaks and Valleys . . . . . . . . . . . . . . . . . . . . . . . 25 3.2.4 Getting Info on markedPP Objects . . . . . . . . . . . . . . . . . . . . . . 25 3.2.5 Exploiting the“Matrix”Features of markedPP Objects . . . . . . . . . . . . 26 14 Extracting Sweeps 26 4.1 cutEvents Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 4.2 Visualizing Sweeps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 4.2.1 MPlot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 4.2.2 show method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 4.2.3 plot method for markedPP objects . . . . . . . . . . . . . . . . . . . . . . 29 4.3 Getting Basic Summary Stats of Sweeps . . . . . . . . . . . . . . . . . . . . . . . 29 5 Getting a Clean Sweep Sub-Sample 31 5.1 setEnvelopePara Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 5.2 selectEvents Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 6 Feature Space Dimension Reduction and Sample Visualization 38 6.1 reduceFeatureSpaceD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 6.2 Exporting Data for GGobi . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 6.3 Exploring Data with GGobi . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 6.3.1 Changing colors and glyphs . . . . . . . . . . . . . . . . . . . . . . . . . . 43 6.3.2 Rotations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 6.3.3 2 D Tour . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 7 Some Examples of GGobi Use 47 7.1 Extra Outliers Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 7.2 Manual Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 7.2.1 Exporting Sorting Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 8 From GGobi to R 60 8.1 Function readClassification . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 8.2 Figuring Out the Correspondence Between GGobi Labels and Levels . . . . . . . . . 60 8.2.1 likeGGobi function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 9 Automatic and Semi-Automatic Clustering 65 9.1 Kmeans Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 9.1.1 Kmeans Clustering Visualization . . . . . . . . . . . . . . . . . . . . . . . . 68 9.2 Bagged Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 9.3 Gaussian Mixture Based Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . 70 9.3.1 Mclust function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 9.3.2 EMclustN function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 9.4 What To Do With All That?: Example of Adjustment . . . . . . . . . . . . . . . . 75 10 Classification 79 10.1 A Simple Approach Coping with Superpositions . . . . . . . . . . . . . . . . . . . . 79 10.1.1 Getting a Labeled Sample . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 10.1.2 Ideal Waveform Construction with Superpositions . . . . . . . . . . . . . . 80 10.1.3 Nearest Neighbor Classification . . . . . . . . . . . . . . . . . . . . . . . . 81 10.1.4 Checking Classification Results . . . . . . . . . . . . . . . . . . . . . . . . 82 10.1.5 Extracting a markedPP Object from Classification Results . . . . . . . . . . 82 10.1.6 Extracting a SpikeTrain Object from Classification Results . . . . . . . . . 85 10.1.7 Updating a modelCenter Model with Results from Classification . . . . . . 85 2A Using Parallel Features 86 A.1 Required Packages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 A.2 Setting some variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 A.3 Starting Up a Snow Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 A.4 A Parallel selectEvents Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 A.5 Parallel Clustering Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 A.5.1 Parallel kmeans clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 A.5.2 Parallel EMclust and EMclustN . . . . . . . . . . . . . . . . . . . . . . . . 88 A.6 Parallel Classification Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 A.6.1 Parallel modelCenter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 A.6.2 Parallel predict . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 A.6.3 Parallel update . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 A.7 Before Quitting R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 31 Introduction What follows is work in progress. I’m working on this new SpikeOMatic version basically everyday so you can expect regular updates. 1.1 Never Used SpikeOMatic before? If you are brand new to SpikeOMatic you have to know that to use what follows you will need to install the R software. It is free, open source and of course really great. You can get it from the R 1project site . You will also need another great, free and open source software: the GGobi Data 2Visualization System . These two software can seem a bit hard to use at first sight. R does not follow the nowadays common“point and click”paradigm. That means that a bit of patience and a careful reading of the tutorials are de rigueur. R documentation is plentiful and goes from the very basic to the most 3advanced stuff. The“contributed documentation”page is good place to start. Look in particular at: “R for Beginners”by Emmanuel Paradis (french and spanish versions are also available) and “An Introduction to R: Software for Statistical Modelling & Computing”by Petra Kuhnert and Bill Venables [12]. Another good place is Ross Ihaka’s course: “Information Visualisation”, the course 4uses R to generate actual examples of data visualization and is a wonderful introduction to its subject. Ross Ihaka, together with Robert Gentleman, is moreover one of the original R developers. 5The“two-day short course in R” of Thomas Lumley form the R Core Development Team is also 6great. There is also a R Wiki site which is worth looking at. 7 WindowsuserscanenjoytheSciViews R GUI developedbyPhilippeGrosjean&EricLecoutre 8and are strongly encouraged to use the Tinn-R editor to edit R codes, etc. Information on how to configure Tinn-R and R can be found in [12]. On Linux I’m using the emacs editor together 9with ESS : Emacs Speaks Statistics . 1.2 About this New Release 10The major change compared to the previous SpikeOMatic version is a full integration with R (not finished yet) and an implementation based on S4 classes and methods developed by John 11Chambers [3] . Don’t get scared reading that if you have no clue about S4 classes and methods, it means, if I don’t completely screw up (!), that SpikeOMatic should be easier to use in an efficient way by anyone. In addition SpikeOMatic makes now use of GGobi . This has two consequences. It should first allow users to develop an“intuitive understanding”of the analysis process. GGobi provides numerous ways to display data (which in our cases will be clusters). These displays can be dynamic and are interactive. Second, it is also possible to perform the clustering“by hand”with GGobi . I do not 1 http://www.r-project.org 2http://www.ggobi.org/ 3http://cra