Numpy beginner's guide

numpy 1.5 beginner's guide pdf and dnumpy beginner's guide second edition pdf
Prof.WilliamsHibbs Profile Pic
Prof.WilliamsHibbs,United States,Teacher
Published Date:28-07-2017
Your Website URL(Optional)
Comment
Guide to NumPy Travis E. Oliphant, PhD Dec 7, 2006 This book is under restricted distribution using a Market-Determined, Tempo- rary,Distribution-Restriction(MDTDR)system(seehttp://www.trelgol.com)until October31,2010atthelatest. Ifyoureceivethisbook, youareaskednottocopyit inanyform(electronicorpaper)untilthetemporarydistribution-restrictionlapses. If you have multiple users at an institution, you should either share a single copy using some form of digital library check-out, or buy multiple copies. The more copies purchased, the sooner the documentation can be released from this incon- venient distribution restriction. After October 31, 2010 this book may be freely copied in any format and used as source material for other books as long as ac- knowledgement of the original author is given. Your support of this temporary distribution restriction playsan essential rolein allowingthe author and otherslike him to produce more quality books and software. 1Contents I NumPy from Python 12 1 Origins of NumPy 13 2 Object Essentials 18 2.1 Data-Type Descriptors . . . . . . . . . . . . . . . . . . . . . . . . . . 19 2.2 Basic indexing (slicing). . . . . . . . . . . . . . . . . . . . . . . . . . 23 2.3 Memory Layout of ndarray . . . . . . . . . . . . . . . . . . . . . . 26 2.3.1 Contiguous Memory Layout . . . . . . . . . . . . . . . . . . . 26 2.3.2 Non-contiguous memory layout . . . . . . . . . . . . . . . . . 28 2.4 Universal Functions for arrays . . . . . . . . . . . . . . . . . . . . . . 30 2.5 Summary of new features . . . . . . . . . . . . . . . . . . . . . . . . 32 2.6 Summary of differences with Numeric . . . . . . . . . . . . . . . . . 34 2.6.1 First-step changes . . . . . . . . . . . . . . . . . . . . . . . . 34 2.6.2 Second-step changes . . . . . . . . . . . . . . . . . . . . . . . 37 2.6.3 Updating code that uses Numeric using alter codeN . . . . . 38 2.6.4 Changes to think about . . . . . . . . . . . . . . . . . . . . . 39 2.7 Summary of differences with Numarray . . . . . . . . . . . . . . . . 40 2.7.1 First-step changes . . . . . . . . . . . . . . . . . . . . . . . . 41 2.7.1.1 Import changes. . . . . . . . . . . . . . . . . . . . . 41 2.7.1.2 Attribute and method changes . . . . . . . . . . . . 42 2.7.2 Second-step changes . . . . . . . . . . . . . . . . . . . . . . . 43 2.7.3 Additional Extension modules. . . . . . . . . . . . . . . . . . 43 3 The Array Object 45 3.1 ndarray Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 3.1.1 Memory Layout attributes . . . . . . . . . . . . . . . . . . . . 45 3.1.2 Data Type attributes . . . . . . . . . . . . . . . . . . . . . . 49 23.1.3 Other attributes . . . . . . . . . . . . . . . . . . . . . . . . . 50 3.1.4 Array Interface attributes . . . . . . . . . . . . . . . . . . . . 52 3.2 ndarray Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 3.2.1 Array conversion . . . . . . . . . . . . . . . . . . . . . . . . . 54 3.2.2 Array shape manipulation . . . . . . . . . . . . . . . . . . . . 59 3.2.3 Array item selection and manipulation . . . . . . . . . . . . . 61 3.2.4 Array calculation . . . . . . . . . . . . . . . . . . . . . . . . . 65 3.3 Array Special Methods . . . . . . . . . . . . . . . . . . . . . . . . . . 71 3.3.1 Methods for standard library functions . . . . . . . . . . . . . 71 3.3.2 Basic customization . . . . . . . . . . . . . . . . . . . . . . . 72 3.3.3 Container customization . . . . . . . . . . . . . . . . . . . . . 74 3.3.4 Arithmetic customization . . . . . . . . . . . . . . . . . . . . 75 3.3.4.1 Binary . . . . . . . . . . . . . . . . . . . . . . . . . 75 3.3.4.2 In-place . . . . . . . . . . . . . . . . . . . . . . . . . 77 3.3.4.3 Unary operations . . . . . . . . . . . . . . . . . . . 78 3.4 Array indexing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 3.4.1 Basic Slicing . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 3.4.2 Advanced selection . . . . . . . . . . . . . . . . . . . . . . . . 81 3.4.2.1 Integer . . . . . . . . . . . . . . . . . . . . . . . . . 81 3.4.2.2 Boolean . . . . . . . . . . . . . . . . . . . . . . . . . 83 3.4.3 Flat Iterator indexing . . . . . . . . . . . . . . . . . . . . . . 84 4 Basic Routines 85 4.1 Creating arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 4.2 Operations on two or more arrays. . . . . . . . . . . . . . . . . . . . 90 4.3 Printing arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 4.4 Functions redundant with methods . . . . . . . . . . . . . . . . . . 94 4.5 Dealing with data types . . . . . . . . . . . . . . . . . . . . . . . . . 94 5 Additional Convenience Routines 96 5.1 Shape functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 5.2 Basic functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 5.3 Polynomial functions . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 5.4 Set Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 5.5 Array construction using index tricks . . . . . . . . . . . . . . . . . . 112 5.6 Other indexing devices . . . . . . . . . . . . . . . . . . . . . . . . . . 115 5.7 Two-dimensional functions. . . . . . . . . . . . . . . . . . . . . . . . 116 35.8 More data type functions . . . . . . . . . . . . . . . . . . . . . . . . 118 5.9 Functions that behave like ufuncs . . . . . . . . . . . . . . . . . . . . 121 5.10 Miscellaneous Functions . . . . . . . . . . . . . . . . . . . . . . . . . 121 5.11 Utility functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 6 Scalar objects 126 6.1 Attributes of array scalars . . . . . . . . . . . . . . . . . . . . . . . . 127 6.2 Methods of array scalars . . . . . . . . . . . . . . . . . . . . . . . . . 129 6.3 Defining New Types . . . . . . . . . . . . . . . . . . . . . . . . . . . 130 7 Data-type (dtype) Objects 131 7.1 Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131 7.2 Construction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133 7.3 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 8 Standard Classes 138 8.1 Special attributes and methods recognized by NumPy . . . . . . . . 139 8.2 Matrix Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 8.3 Memory-mapped-file arrays . . . . . . . . . . . . . . . . . . . . . . . 142 8.4 Character arrays (numpy.char) . . . . . . . . . . . . . . . . . . . . . 143 8.5 Record Arrays (numpy.rec) . . . . . . . . . . . . . . . . . . . . . . . 144 8.6 Masked Arrays (numpy.ma) . . . . . . . . . . . . . . . . . . . . . . . 148 8.7 Standard container class . . . . . . . . . . . . . . . . . . . . . . . . . 149 8.8 Array Iterators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149 8.8.1 Default iteration . . . . . . . . . . . . . . . . . . . . . . . . . 150 8.8.2 Flat iteration . . . . . . . . . . . . . . . . . . . . . . . . . . . 150 8.8.3 N-dimensional enumeration . . . . . . . . . . . . . . . . . . . 151 8.8.4 Iterator for broadcasting . . . . . . . . . . . . . . . . . . . . . 151 9 Universal Functions 153 9.1 Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 9.1.1 Broadcasting . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 9.1.2 Output type determination . . . . . . . . . . . . . . . . . . . 154 9.1.3 Use of internal buffers . . . . . . . . . . . . . . . . . . . . . . 155 9.1.4 Error handling . . . . . . . . . . . . . . . . . . . . . . . . . . 155 9.1.5 Optional keyword arguments . . . . . . . . . . . . . . . . . . 156 9.2 Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 9.3 Casting Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158 49.4 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 9.4.1 Reduce . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 9.4.2 Accumulate . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 9.4.3 Reduceat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162 9.4.4 Outer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163 9.5 Available ufuncs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164 9.5.1 Math operations . . . . . . . . . . . . . . . . . . . . . . . . . 164 9.5.2 Trigonometric functions . . . . . . . . . . . . . . . . . . . . . 167 9.5.3 Bit-twiddling functions. . . . . . . . . . . . . . . . . . . . . . 168 9.5.4 Comparison functions . . . . . . . . . . . . . . . . . . . . . . 169 9.5.5 Floating functions . . . . . . . . . . . . . . . . . . . . . . . . 171 10 Basic Modules 174 10.1 Linear Algebra (linalg) . . . . . . . . . . . . . . . . . . . . . . . . 174 10.2 Discrete Fourier Transforms (fft) . . . . . . . . . . . . . . . . . . . 177 10.3 Random Numbers (random) . . . . . . . . . . . . . . . . . . . . . . 181 10.3.1 Discrete Distributions . . . . . . . . . . . . . . . . . . . . . . 181 10.3.2 Continuous Distributions . . . . . . . . . . . . . . . . . . . . 184 10.3.3 Miscellaneous utilities . . . . . . . . . . . . . . . . . . . . . . 190 10.4 Matrix-specific functions (matlib) . . . . . . . . . . . . . . . . . . . . 191 10.5 Ctypes utiltity functions (ctypeslib) . . . . . . . . . . . . . . . . . . 191 11 Testing and Packaging 192 11.1 Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192 11.2 NumPy Distutils . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194 11.2.1 misc util . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194 11.2.2 Other modules . . . . . . . . . . . . . . . . . . . . . . . . . . 202 11.3 Conversion of .src files . . . . . . . . . . . . . . . . . . . . . . . . . . 203 11.3.1 Fortran files . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203 11.3.1.1 Named repeat rule . . . . . . . . . . . . . . . . . . . 204 11.3.1.2 Short repeat rule . . . . . . . . . . . . . . . . . . . . 204 11.3.1.3 Pre-defined names . . . . . . . . . . . . . . . . . . . 204 11.3.2 Other files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205 II C-API 206 12 New Python Types and C-Structures 207 512.1 New Python Types Defined . . . . . . . . . . . . . . . . . . . . . . . 208 12.1.1 PyArray Type . . . . . . . . . . . . . . . . . . . . . . . . . . 209 12.1.2 PyArrayDescr Type . . . . . . . . . . . . . . . . . . . . . . . 210 12.1.3 PyUFunc Type . . . . . . . . . . . . . . . . . . . . . . . . . . 218 12.1.4 PyArrayIter Type . . . . . . . . . . . . . . . . . . . . . . . . 220 12.1.5 PyArrayMultiIter Type . . . . . . . . . . . . . . . . . . . . . 222 12.1.6 PyArrayFlags Type . . . . . . . . . . . . . . . . . . . . . . . 223 12.1.7 ScalarArrayTypes . . . . . . . . . . . . . . . . . . . . . . . . 223 12.2 Other C-Structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224 12.2.1 PyArray Dims . . . . . . . . . . . . . . . . . . . . . . . . . . 224 12.2.2 PyArray Chunk. . . . . . . . . . . . . . . . . . . . . . . . . . 224 12.2.3 PyArrayInterface . . . . . . . . . . . . . . . . . . . . . . . . . 225 12.2.4 Internally used structures . . . . . . . . . . . . . . . . . . . . 226 12.2.4.1 PyUFuncLoopObject . . . . . . . . . . . . . . . . . 227 12.2.4.2 PyUFuncReduceObject . . . . . . . . . . . . . . . . 227 12.2.4.3 PyUFunc Loop1d . . . . . . . . . . . . . . . . . . . 227 12.2.4.4 PyArrayMapIter Type . . . . . . . . . . . . . . . . 227 13 Complete API 228 13.1 Configuration defines . . . . . . . . . . . . . . . . . . . . . . . . . . . 228 13.1.1 Guaranteed to be defined . . . . . . . . . . . . . . . . . . . . 228 13.1.2 Possible defines . . . . . . . . . . . . . . . . . . . . . . . . . . 229 13.2 Array Data Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229 13.2.1 Enumerated Types . . . . . . . . . . . . . . . . . . . . . . . . 230 13.2.2 Defines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231 13.2.2.1 Max and min values for integers . . . . . . . . . . . 231 13.2.2.2 Number of bits in data types . . . . . . . . . . . . . 231 13.2.2.3 Bit-width references to enumerated typenums. . . . 231 13.2.2.4 Integer that can hold a pointer . . . . . . . . . . . . 232 13.2.3 C-type names . . . . . . . . . . . . . . . . . . . . . . . . . . . 232 13.2.3.1 Boolean . . . . . . . . . . . . . . . . . . . . . . . . . 232 13.2.3.2 (Un)Signed Integer . . . . . . . . . . . . . . . . . . 232 13.2.3.3 (Complex) Floating point . . . . . . . . . . . . . . . 233 13.2.3.4 Bit-width names . . . . . . . . . . . . . . . . . . . . 233 13.2.4 Printf Formatting . . . . . . . . . . . . . . . . . . . . . . . . 233 13.3 Array API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233 13.3.1 Array structure and data access . . . . . . . . . . . . . . . . 233 613.3.1.1 Data access . . . . . . . . . . . . . . . . . . . . . . . 235 13.3.2 Creating arrays . . . . . . . . . . . . . . . . . . . . . . . . . . 236 13.3.2.1 From scratch . . . . . . . . . . . . . . . . . . . . . . 236 13.3.2.2 From other objects. . . . . . . . . . . . . . . . . . . 238 13.3.3 Dealing with types . . . . . . . . . . . . . . . . . . . . . . . . 244 13.3.3.1 General check of Python Type . . . . . . . . . . . . 244 13.3.3.2 Data-type checking . . . . . . . . . . . . . . . . . . 246 13.3.3.3 Converting data types . . . . . . . . . . . . . . . . . 249 13.3.3.4 New data types . . . . . . . . . . . . . . . . . . . . 251 13.3.3.5 Special functions for PyArray OBJECT . . . . . . . 252 13.3.4 Array flags . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253 13.3.4.1 Basic Array Flags . . . . . . . . . . . . . . . . . . . 253 13.3.4.2 Combinations of array flags . . . . . . . . . . . . . . 254 13.3.4.3 Flag-like constants . . . . . . . . . . . . . . . . . . 254 13.3.4.4 Flag checking . . . . . . . . . . . . . . . . . . . . . . 255 13.3.5 Array method alternative API . . . . . . . . . . . . . . . . . 256 13.3.5.1 Conversion . . . . . . . . . . . . . . . . . . . . . . . 256 13.3.5.2 Shape Manipulation . . . . . . . . . . . . . . . . . . 258 13.3.5.3 Item selection and manipulation . . . . . . . . . . . 260 13.3.5.4 Calculation . . . . . . . . . . . . . . . . . . . . . . . 263 13.3.6 Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265 13.3.6.1 Array Functions . . . . . . . . . . . . . . . . . . . . 265 13.3.6.2 Other functions . . . . . . . . . . . . . . . . . . . . 267 13.3.7 Array Iterators . . . . . . . . . . . . . . . . . . . . . . . . . . 268 13.3.8 Broadcasting (multi-iterators) . . . . . . . . . . . . . . . . . . 269 13.3.9 Array Scalars . . . . . . . . . . . . . . . . . . . . . . . . . . . 271 13.3.10Data-type descriptors . . . . . . . . . . . . . . . . . . . . . . 273 13.3.11Conversion Utilities . . . . . . . . . . . . . . . . . . . . . . . 275 13.3.11.1For use with PyArg ParseTuple . . . . . . . . . . 275 13.3.11.2Other conversions . . . . . . . . . . . . . . . . . . . 277 13.3.12Miscellaneous . . . . . . . . . . . . . . . . . . . . . . . . . . . 278 13.3.12.1Importing the API . . . . . . . . . . . . . . . . . . . 278 13.3.12.2Internal Flexibility . . . . . . . . . . . . . . . . . . . 279 13.3.12.3Memory management . . . . . . . . . . . . . . . . . 280 13.3.12.4Threading support . . . . . . . . . . . . . . . . . . . 280 13.3.12.5Priority . . . . . . . . . . . . . . . . . . . . . . . . . 282 13.3.12.6Default buffers . . . . . . . . . . . . . . . . . . . . . 282 713.3.12.7Other constants . . . . . . . . . . . . . . . . . . . . 282 13.3.12.8Miscellaneous Macros . . . . . . . . . . . . . . . . . 283 13.3.12.9Enumerated Types . . . . . . . . . . . . . . . . . . . 284 13.4 UFunc API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284 13.4.1 Constants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284 13.4.2 Macros . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285 13.4.3 Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285 13.4.4 Generic functions . . . . . . . . . . . . . . . . . . . . . . . . . 288 13.5 Importing the API . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290 14 How to extend NumPy 292 14.1 Writing an extension module . . . . . . . . . . . . . . . . . . . . . . 292 14.2 Required subroutine . . . . . . . . . . . . . . . . . . . . . . . . . . . 293 14.3 Defining functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294 14.3.1 Functions without keyword arguments . . . . . . . . . . . . . 295 14.3.2 Functions with keyword arguments . . . . . . . . . . . . . . . 296 14.3.3 Reference counting . . . . . . . . . . . . . . . . . . . . . . . . 297 14.4 Dealing with array objects . . . . . . . . . . . . . . . . . . . . . . . . 298 14.4.1 Converting an arbitrary sequence object . . . . . . . . . . . . 299 14.4.2 Creating a brand-new ndarray . . . . . . . . . . . . . . . . . 301 14.4.3 Getting at ndarray memory and accessing elements of the ndarray . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303 14.5 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303 15 Beyond the Basics 305 15.1 Iterating over elements in the array . . . . . . . . . . . . . . . . . . . 305 15.1.1 Basic Iteration . . . . . . . . . . . . . . . . . . . . . . . . . . 305 15.1.2 Iterating over all but one axis . . . . . . . . . . . . . . . . . . 307 15.1.3 Iterating over multiple arrays . . . . . . . . . . . . . . . . . . 307 15.1.4 Broadcasting over multiple arrays . . . . . . . . . . . . . . . 308 15.2 Creating a new universal function . . . . . . . . . . . . . . . . . . . . 309 15.3 User-defined data-types . . . . . . . . . . . . . . . . . . . . . . . . . 312 15.3.1 Adding the new data-type . . . . . . . . . . . . . . . . . . . . 312 15.3.2 Registering a casting function . . . . . . . . . . . . . . . . . . 313 15.3.3 Registering coercion rules . . . . . . . . . . . . . . . . . . . . 314 15.3.4 Registering a ufunc loop . . . . . . . . . . . . . . . . . . . . . 315 15.4 Subtyping the ndarray in C . . . . . . . . . . . . . . . . . . . . . . . 315 815.4.1 Creating sub-types . . . . . . . . . . . . . . . . . . . . . . . . 316 15.4.2 Specific features of ndarray sub-typing . . . . . . . . . . . . . 317 15.4.2.1 The array finalize method . . . . . . . . . . . . . 317 15.4.2.2 The array priority attribute . . . . . . . . . . . . 318 15.4.2.3 The array wrap method . . . . . . . . . . . . . . 318 16 Using Python as glue 319 16.1 Calling other compiled libraries from Python . . . . . . . . . . . . . 320 16.2 Hand-generated wrappers . . . . . . . . . . . . . . . . . . . . . . . . 320 16.3 f2py . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321 16.3.1 Creating source for a basic extension module . . . . . . . . . 321 16.3.2 Creating a compiled extension module . . . . . . . . . . . . . 322 16.3.3 Improving the basic interface . . . . . . . . . . . . . . . . . . 323 16.3.4 Inserting directives in Fortran source . . . . . . . . . . . . . . 324 16.3.5 A filtering example . . . . . . . . . . . . . . . . . . . . . . . . 325 16.3.6 Calling f2py from Python . . . . . . . . . . . . . . . . . . . . 326 16.3.7 Automatic extension module generation . . . . . . . . . . . . 326 16.3.8 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327 16.4 weave . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 328 16.4.1 Speed up code involving arrays (also see scipy.numexpr) . . . 328 16.4.2 Inline C-code . . . . . . . . . . . . . . . . . . . . . . . . . . . 329 16.4.3 Simplify creation of an extension module . . . . . . . . . . . 331 16.4.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332 16.5 Pyrex . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332 16.5.1 Pyrex-add . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333 16.5.2 Pyrex-filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335 16.5.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336 16.6 ctypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337 16.6.1 Having a shared library . . . . . . . . . . . . . . . . . . . . . 338 16.6.2 Loading the shared library . . . . . . . . . . . . . . . . . . . 339 16.6.3 Converting arguments . . . . . . . . . . . . . . . . . . . . . . 340 16.6.4 Calling the function . . . . . . . . . . . . . . . . . . . . . . . 341 16.6.5 Complete example . . . . . . . . . . . . . . . . . . . . . . . . 342 16.6.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 346 16.7 Additional tools you may find useful . . . . . . . . . . . . . . . . . . 347 16.7.1 SWIG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347 16.7.2 SIP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 348 916.7.3 Boost Python . . . . . . . . . . . . . . . . . . . . . . . . . . . 348 16.7.4 Instant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349 16.7.5 PyInline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350 16.7.6 PyFort . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350 17 Code Explanations 351 17.1 Memory model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351 17.2 Data-type encapsulation . . . . . . . . . . . . . . . . . . . . . . . . 352 17.3 N-D Iterators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 352 17.4 Broadcasting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353 17.5 Array Scalars . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354 17.6 Advanced (“Fancy”) Indexing . . . . . . . . . . . . . . . . . . . . . . 354 17.6.1 Fancy-indexing check. . . . . . . . . . . . . . . . . . . . . . . 355 17.6.2 Fancy-indexing implementation . . . . . . . . . . . . . . . . . 356 17.6.2.1 Creating the mapping object . . . . . . . . . . . . . 356 17.6.2.2 Binding the mapping object . . . . . . . . . . . . . 356 17.6.2.3 Getting (or Setting) . . . . . . . . . . . . . . . . . . 357 17.7 Universal Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . 357 17.7.1 Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357 17.7.2 Function call . . . . . . . . . . . . . . . . . . . . . . . . . . . 359 17.7.2.1 One Loop . . . . . . . . . . . . . . . . . . . . . . . . 359 17.7.2.2 Strided Loop . . . . . . . . . . . . . . . . . . . . . . 360 17.7.2.3 Buffered Loop . . . . . . . . . . . . . . . . . . . . . 360 17.7.3 Final output manipulation. . . . . . . . . . . . . . . . . . . . 361 17.7.4 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361 17.7.4.1 Setup . . . . . . . . . . . . . . . . . . . . . . . . . . 361 17.7.4.2 Reduce . . . . . . . . . . . . . . . . . . . . . . . . . 362 17.7.4.3 Accumulate . . . . . . . . . . . . . . . . . . . . . . . 363 17.7.4.4 Reduceat . . . . . . . . . . . . . . . . . . . . . . . . 363 10List of Tables 2.1 Built-in array-scalar types corresponding to data-types for an ndar- ray. The bold-face types correspond to standard Python types. The object type is special because arrays with dtype=’O’ do not return an array scalar on item access but instead return the actual object referenced in the array. . . . . . . . . . . . . . . . . . . . . . . . . . . 22 3.1 Attributes of the ndarray. . . . . . . . . . . . . . . . . . . . . . . . 46 3.2 Array conversion methods . . . . . . . . . . . . . . . . . . . . . . . . 58 3.3 Array item selection and shape manipulation methods. If axis is an argument, then the calculationis performedalongthat axis. An axis value of None means the array is flattened before calculation proceeds. 66 3.4 Array object calculation methods. If axis is an argument, then the calculationisperformedalongthataxis. AnaxisvalueofNonemeans the array is flattened before calculation proceeds. All of these meth- odscantakeanoptionalout=argumentwhichcanspecifytheoutput array to write the results into. . . . . . . . . . . . . . . . . . . . . . 70 6.1 Array scalar types that inherit from basic Python types. The intc arraydatatypemightalsoinheritfromtheIntTypeifithasthesame number of bits as the int array data type on your platform. . . . . . 127 9.1 Universal function (ufunc) attributes. . . . . . . . . . . . . . . . . . 158 10.1 Functions in numpy.dual (both in NumPy and SciPy) . . . . . . . . 174 11Part I NumPy from Python 12Chapter 1 Origins of NumPy NumPy builds on (and is a successor to) the successful Numeric array object. Its goal is to create the corner-stone for a useful environment for scientific computing. In order to better understand the people surrounding NumPy and (its library- package) SciPy, I will explain a little about how SciPy and (current) NumPy orig- inated. In 1998, as a graduate student studying biomedical imaging at the Mayo Clinic in Rochester, MN, I came across Python and its numerical extension (Nu- meric) while I was looking for ways to analyze large data sets for Magnetic Res- onance Imaging and Ultrasound using a high-level language. I quickly fell in love with Python programming which is a remarkable statement to make about a pro- gramming language. If I had not seen others with the same view, I might have seriouslydoubted mysanity. I became ratherinvolvedin the Numeric Pythoncom- munity, adding the C-API chapter to the Numeric documentation (for which Paul Dubois graciously made me a co-author). As I progressed with my thesis work, programming in Python was so enjoyable thatIfeltinhibitedwhenIworkedwithotherprogrammingframeworks. Asaresult, when a task I needed to perform was not available in the core language, or in the Numeric extension, I looked around and found C or Fortran code that performed the needed task, wrapped it into Python (either by hand orusing SWIG), and used the new functionality in my programs. Alongthe way,I learnedagreatdeal aboutthe underlyingstructureof Numeric andgrewtoadmireit’ssimplebutelegantstructuresthatgrewoutofthemechanism by which Python allows itself to be extended. 13NOTE Numeric was originally written in 1995 largely by Jim Hugunin while he was a graduate student at MIT. He received help from many people including Jim Fulton, David Ascher, Paul Dubois, and Konrad Hinsen. These individuals and many others added comments, criticisms, and code which helped the Numeric exten- sion reach stability. Jim Hugunin did not stay long as an active memberofthecommunity—movingontowriteJythonand,later, Iron Python. By operating in this need-it-make-it fashion I ended up with a substantial li- brary of extension modules that helped Python + Numeric become easier to use in a scientific setting. These early modules included raw input-output functions, a special function library, an integration library, an ordinary differential equation solver, some least-squaresoptimizers, and sparse matrix solvers. While I was doing this laborious work, Pearu Peterson noticed that a lot of the routines I was wrap- ping were written in Fortran and there was no simplified wrapping mechanism for Fortran subroutines (like SWIG for C). He began the task of writing f2py which made it possible to easilywrap Fortranprogramsinto Python. I helped him alittle bit, mostly with testing and contributing early function-call-back code, but he put forth the brunt of the work. His result was simply amazingto me. I’ve alwaysbeen impressed with f2py, especially because I knew how much effort writing and main- taining extension modules could be. Anybody serious about scientific computing with Python will appreciate that f2py is distributed along with NumPy. When I finished my Ph.D. in 2001, Eric Jones (who had recently completed his Ph.D.atDuke)contactedmebecausehehadacollectionofPythonmoduleshehad developedaspartofhisthesisworkaswell. Hewantedtocombinehismoduleswith mine into one super package. Together with Pearu Peterson we joined our efforts, and SciPy was born in 2001. Since then, many people have contributed module code to SciPy including Ed Schofield, Robert Cimrman, David M. Cooke, Charles (Chuck)Harris,PrabhuRamachandran,GaryStrangman,Jean-SebastienRoy,and Fernando Perez. Others such as Travis Vaught, David Morrill, Jeff Whitaker, and Louis Luangkesorn have contributed testing and build support. At the start of 2005, SciPy was at release 0.3 and relatively stable for an early version number. Part of the reason it was difficult to stabilize SciPy was that the arrayobjectuponwhichSciPybuildswasundergoingabitofanupheaval. Atabout 14thesametimeasSciPywasbeingbuilt, someNumericuserswerehittingupagainst the limited capabilities of Numeric. In particular, the ability to deal with memory mapped files (and associated alignment and swapping issues), record arrays, and alterederrorcheckingmodeswereimportantbutlimitedornon-existentinNumeric. Asaresult,numarraywascreatedbyPerryGreenfield,ToddMiller,andRickWhite at the Space Science Telescope Institute as a replacement for Numeric. Numarray used a very different implementation scheme as a mix of Python classes and C code (which led to slow downs in certain common uses). While improving some capabilities, it was slow to pick up on the more advanced features of Numeric’s universal functions (ufuncs) — never re-creating the C-API that SciPy depended on. This made it difficult for SciPy to “convert” to numarray. Many newcomers to scientific computing with Python were told that numarray was the future and started developing for it. Very useful tools were developed that could not be used with Numeric (because of numarray’s change in C-API), and therefore could not be used easily in SciPy. This state of affairs was very discouragingformepersonallyasitleftthecommunityfragmented. Somedeveloped fornumarray,othersdevelopedaspartofSciPy. Afewpeopleevenrejectedadopting Pythonforscientificcomputingentirelybecauseofthe split. Inaddition, Iestimate that quite a few Python users simply stayed away from both SciPy and numarray, leaving the community smaller than it could have been given the number of people that use Python for science and engineering purposes. It should be recognized that the split was not intentional, but simply an out- growth of the different and exacting demands of scientific computing users. My describing these events should not be construed as assigning blame to anyone. I very much admire and appreciate everyone I’ve met who is involved with scientific computing and Python. Using a stretched biological metaphor, it is only through theprocessofdividingandmergingthatbetterresultsareborn. Ithinkthisconcept applies to NumPy. Inearly2005,Idecidedtobeginanefforttohelpbringthedivergingcommunity together under a common framework if it were possible. I first looked at numarray to see what could be done to add the missing features to make SciPy work with it as a core array object. After a couple of days of studying numarray, I was not enthusiastic about this approach. My familiarity with the Numeric code base no doubt biased my opinion, but it seemed to me that the features of Numarray could be added backto Numeric with afew fundamental changesto the coreobject. This would make the transition of SciPy to a more enhanced array object much easier in my mind. 15Therefore, I began to construct this hybrid array object complete with an en- hanced set of universal (broadcasting) functions that could deal with it. Along the way,quite afew newfeaturesandsignificantenhancementswereaddedtothearray object and its surrounding infrastructure. This book describes the result of that year-and-a-half-long effort which culminated with the release of NumPy 0.9.2 in early2006and NumPy 1.0in late2006. I firstnamed the new package,SciPyCore, and used the scipy namespace. However, after a few months of testing under that name, it became clear that a separate namespace was needed for the new package. As a result, a rapid search for a new name resulted in actually coming back to the NumPy name which was the unofficial name of Numerical Python but never the actual namespace. Because the new package builds on the code-base of and is a successor to Numeric, I think the NumPy name is fitting and hopefully not too confusing to new users. This book only briefly outlines some of the infrastructure that surrounds the basicobjectsinNumPytoprovidetheadditionalfunctionalitycontainedintheolder Numeric package (i.e. LinearAlgebra, RandomArray, FFT). This infrastructure in NumPy includes basic linear algebra routines, Fourier transform capabilities, and random number generators. In addition, the f2py module is described in its own documentation, and so is only briefly mentioned in the second part of the book. There are also extensions to the standard Python distutils and testing frameworks includedwithNumPythatareusefulinconstructingyourownpackagesbuiltontop of NumPy. The central purpose of this book, however,is to describe and document the basic NumPy system that is available under the numpy namespace. NOTE The numpy namespace includes all names under the numpy.core and numpy.lib namespaces as well. Thus, import numpy will alsoimport the namesfromnumpy.coreandnumpy.lib. Thisis the recommended way to use numpy. Thefollowingtable givesa brief outline ofthe sub-packagescontainedin numpy package. 16Sub-Package Purpose Comments core basic objects all names exported to numpy lib additional utilities all names exported to numpy linalg basic linear algebra old LinearAlgebra from Numeric fft discrete Fourier transforms old FFT from Numeric random random number generators old RandomArray from Numeric distutils enhanced build and distribution improvements built on standard distutils testing unit-testing utility functions useful for testing f2py automatic wrapping of Fortran code a useful utility needed by SciPy 17Chapter 2 Object Essentials NumPy provides two fundamental objects: an N-dimensional array object (ndarray) and a universal function object (ufunc). In addition, there are other objectsthatbuildontopofthesewhichyoumayfindusefulinyourwork,andthese will be discussed later. The current chapter will provide background information on just the ndarray and the ufunc that will be important for understanding the attributes and methods to be discussed later. AnN-dimensionalarrayisahomogeneouscollectionof“items”indexedusingN integers. TherearetwoessentialpiecesofinformationthatdefineanN-dimensional array: 1) the shape of the array, and 2) the kind of item the array is composed of. The shape of the array is a tuple of N integers (one for each dimension) that provides information on how far the index can vary along that dimension. The other important information describing an array is the kind of item the array is composed of. Because every ndarray is a homogeneous collection of exactly the same data-type, everyitem takes up the samesize block of memory, and each block 1 of memory in the array is interpreted in exactly the same way . TIP i All arrays in NumPy are indexed starting at 0 and ending at M-1 following the Python convention. For example, consider the following piece of code: 1 By using OBJECT arrays, one can effectively have heterogeneous arrays, but the system still sees each element of the array as exactly the same thing (a reference to a Python object). 18 a = array(1,2,3,4,5,6) a.shape (2, 3) a.dtype dtype(’int32’) NOTE for all code in this book it is assumed that you have first entered from numpy import . In addition, any previously defined ar- rays are still defined for subsequent examples. Thiscodedefinesanarrayofsize2×3composedof4-byte(little-endian)integer elements (on my 32-bit platform). We can index into this two-dimensional array using two integers: the first integer running from 0 to 1 inclusive and the second from 0 to 2 inclusive. For example, index (1,1) selects the element with value 5: a1,1 5 All code shown in the shaded-boxes in this book has been (automatically) exe- cutedonaparticularversionofNumPy. Theoutputofthecodeshownbelowshows which version of NumPy was used to create all of the output in your copy of this book. import numpy; print numpy. version 1.0.2.dev3478 2.1 Data-Type Descriptors In NumPy, an ndarray is an N-dimensional array of items where each item takes up a fixed number of bytes. Typically, this fixed number of bytes represents a number (e.g. integer or floating-point). However, this fixed number of bytes could also represent an arbitrary record made up of any collection of other data types. NumPyachievesthisflexibilitythroughtheuseofadata-type(dtype)object. Every array has an associated dtype object which describes the layout of the array data. Every dtype object, in turn, has an associated Python type-object that determines 19head data−type array scalar header ... ndarray Figure 2.1: Conceptual diagram showing the relationship between the three fun- damental objects used to describe the data in an array: 1) the ndarray itself, 2) the data-type object that describes the layout of a single fixed-size element of the array, 3) the array-scalar Python object that is returned when a single element of the array is accessed. exactly what type of Python object is returned when an element of the array is accessed. The dtype objects are flexible enough to contain references to arrays of other dtype objects and, therefore, can be used to define nested records. This advanced functionality will be described in better detail later as it is mainly useful for the recarray(record array)subclass that will also be defined later. However, all ndarrayscanenjoytheflexibilityprovidedbythedtypeobjects. Figure2.1provides a conceptual diagram showing the relationship between the ndarray, its associated data-type object, and an array-scalarthat is returned when a single-element of the array is accessed. Note that the data-type points to the type-object of the array scalar. An array scalar is returned using the type-object and a particular element of the ndarray. Every dtype object is based on one of 21 built-in dtype objects. These built- in objects allow numeric operations on a wide-variety of integer, floating-point, and complex data types. Associated with each data-type is a Python type object whoseinstancesarearrayscalars. Thistype-objectcanbe obtainedusingthetype attribute of the dtype object. Python typically defines only one data-type of a particular data class (one integer type, one floating-point type, etc.). This can be convenient for some applications that don’t need to be concerned with all the ways data can be represented in a computer. For scientific applications, however, this is not always true. As a result, in NumPy, their are 21 different fundamental Python data-type-descriptor objects built-in. These descriptors are mostly based on the types available in the C language that CPython is written in. However, there are a 20

Advise: Why You Wasting Money in Costly SEO Tools, Use World's Best Free SEO Tool Ubersuggest.