Big Data Application in Power Systems

Opis

Big Data Application in Power Systems brings together experts from academia, industry and regulatory agencies who share their understanding and discuss the big data analytics applications for power systems diagnostics, operation and control. Recent developments in monitoring systems and sensor networks dramatically increase the variety, volume and velocity of measurement data in electricity transmission and distribution level. The book focuses on rapidly modernizing monitoring systems, measurement data availability, big data handling and machine learning approaches to process high dimensional, heterogeneous and spatiotemporal data. The book chapters discuss challenges, opportunities, success stories and pathways for utilizing big data value in smart grids.

Provides expert analysis of the latest developments by global authorities
Contains detailed references for further reading and extended research
Provides additional cross-disciplinary lessons learned from broad disciplines such as statistics, computer science and bioinformatics
Focuses on rapidly modernizing monitoring systems, measurement data availability, big data handling and machine learning approaches to process high dimensional, heterogeneous and spatiotemporal data

Spis treści

Front Cover 2
Big Data Application in Power Systems 5
Copyright 6
Contents 7
Contributors 13
About the Editors 15
Preface: Objective and Overview of the Book 17
- Section One: Harness the Big Data From Power Systems 18
- Section Two: Harness the Power of Big Data 21
- Section Three: Put the Power of Big Data Into Power Systems 23
Acknowledgments 27
Section 1: Harness the Big Data From Power Systems 29
- Chapter 1: A Holistic Approach to Becoming a Data-Driven Utility 31
  - 1. Introduction 32
  - 2. Aligning Internal and External Stakeholders 32

Front Cover 2
Big Data Application in Power Systems 5
Copyright 6
Contents 7
Contributors 13
About the Editors 15
Preface: Objective and Overview of the Book 17
- Section One: Harness the Big Data From Power Systems 18
- Section Two: Harness the Power of Big Data 21
- Section Three: Put the Power of Big Data Into Power Systems 23
Acknowledgments 27
Section 1: Harness the Big Data From Power Systems 29
- Chapter 1: A Holistic Approach to Becoming a Data-Driven Utility 31
  - 1. Introduction 32
  - 2. Aligning Internal and External Stakeholders 32
  - 3. Taking a Holistic Approach 33
  - 4. ``Strong´´ First, Then ``Smart´´ 34
  - 5. Increasing Visibility With IEDs 36
  - 6. Network Response Requirements 38
  - 7. Integration Before Automation 39
  - 8. Functional Data Paths: Keep it Simple 40
  - 9. From Sensor to End User: The Process 41
  - 10. Consumers/Customers: Another Source of Data 44
  - 11. Extracting Value From Data, and Presenting It 46
  - 13. Three Case Studies 47
    - 13.1. Frankfort, Kentucky, and Greenfield SCADA, SA 48
    - 13.2. Ketchikan, Alaska, Deals With Unsupported, Legacy RTUs 49
    - 13.3. North Carolina Agency Pursues New SCADA, Boosts Revenue 51
  - 12. The Transformation 47
  - 14. Conclusion 53
    - 14.1. Looking Ahead 54
  - References 55
- Chapter 2: Emerging Security and Data Privacy Challenges for Utilities: Case Studies and Solutions 57
  - 1. Introduction 57
  - 2. Case Studies: The State and Scope of the Threat 58
    - 2.1. Coordinated Cyberattack Causes Outage in the Ukraine 59
    - 2.2. Severe Financial Impacts at Saudi Aramco 60
    - 2.3. The Misunderstood Near Miss: Burlington Electric and Grizzly Steppe 61
    - 2.4. Impact on Practices in the Utility Industry 62
  - 3. The Digitized Network Increases Vulnerability 63
    - 3.1. Attack Scenarios 64
  - 4. The Role of Data Analytics 65
    - 4.1. The Role of Privacy 67
  - 5. Conclusion 68
  - References 69
- Chapter 3: The Role of Big Data and Analytics in Utility Innovation 71
  - 1. Introduction of Big Data and Analytics as an Accelerator of Innovation 71
  - 2. Approaches to Data Driven Innovation 73
  - 3. Integration of Renewable Energy 75
  - 4. Grid Operations 76
  - 5. Cognitive Computing on Big Data 79
  - 6. Weather, the Biggest Data Topic for Power Systems 82
  - References 83
  - Further Reading 84
- Chapter 4: Frameworks for Big Data Integration, Warehousing, and Analytics 85
  - 1. Introduction 85
  - 2. Frameworks for Big Data Platform 86
    - 2.1. Architecture 88
    - 2.2. Storage 89
    - 2.3. Security 89
  - 3. Big Data With HPC 89
    - 3.1. System Architecture (Fig. 3) 90
    - 3.2. Service Type 90
    - 3.3. Internet of Things 91
    - 3.4. HPC Platform for Smart Grid 92
  - 4. Big Data With Complex Event Processing 93
  - 5. Application of Big Data Techniques for Power Systems 97
  - Acknowledgment 100
  - References 100
  - 6. Conclusion 100
  - Further Reading 101
- Section 2: Harness the Power of Big data 103
  - Chapter 5: Moving Toward Agile Machine Learning for Data Analytics in Power Systems 105
    - 1. Introduction 105
    - 2. Classic Supervised and Unsupervised Learning 107
      - 2.1. Supervised Learning Overview 107
        
        2.1.1. Examples of Supervised Learning as a Regularization Empirical Risk Minimization 108
        
        Linear/Ridge Regression 108
        
        Logistic Regression 109
        
        Support Vector Machine 109
        
        Nonparametric Regression 110
        
        Decision Tree 111
        
        2.1.2. Bayesian Perspectives 111
        
        Example: HMM With Autoregressive Emissions 112
        
        Linear/Ridge Regression Revisited 112
        
        2.2. Unsupervised Learning Overview 115
        
        Principle Component Analysis 115
        
        k-Means Clustering 116
        
        3. Model Selection (Hyperparameter Selection) 118
        
        3.1. Theoretical Intuition 119
        
        Generalization Bound 119
        
        Bias-Variance Trade-Off 120
        
        3.2. Practical Model Selection Techniques 120
        
        Cross-Validation 120
        
        Being a Bayesian 122
        
        Regularization Path Algorithms 122
        
        Bayesian Optimization 122
        
        4. Feature Selection 123
        
        4.1. Overview 123
        
        Wrapper Method 125
        
        Embedded Method 125
        
        Filter Method 125
        
        4.2. An Practical Example of FS Using Information Theory Criterion 125
        
        Single Stream Features Extraction 127
        
        Interstream Features 128
        
        5. Other Promising Research Directions 130
        
        Leverage Unlabeled Data With Semisupervised Learning 130
        
        Knowledge Transfer With Multitask Learning 130
        
        Information Fusion With Multiview Learning 131
        
        Other Promising Directions 131
        
        References 132
        
        Chapter 6: Unsupervised Learning Methods for Power System Data Analysis 135
        
        1. Introduction 135
        
        2. Smart Meter Data Preparation 136
        
        2.1. Statistical Analysis 138
        
        3. Clustering Algorithm 139
        
        4. Clustering Approach and Visualization 141
        
        4.1. Features Extraction 141
        
        4.2. Typical Daily Patterns 143
        
        4.3. Visualization Tool 145
        
        5. Conclusions 150
        
        References 152
        
        Chapter 7: Deep Learning for Power System Data Analysis 153
        
        1. Introduction 153
        
        1.1. From Neural Network Towards Deep Learning 153
        
        1.2. Deep Learning Methods 155
        
        2. Supervised Energy Prediction Using Deep Learning 155
        
        2.1. Conditional Restricted Boltzmann Machine 157
        
        2.1.1. Inference in CRBM 158
        
        2.1.2. Learning for CRBM Using Contrastive Divergence 159
        
        2.2. Factored Conditional Restricted Boltzmann Machine 160
        
        2.2.1. Total Energy for FCRBM 161
        
        2.2.2. Inference in FCRBM 162
        
        2.2.3. Learning and Update Rules for FCRBMs 162
        
        2.3. Experiments and Results 163
        
        3. Unsupervised Energy Prediction Using Deep Learning 167
        
        3.1. Problem Formulation 168
        
        3.2. Reinforcement Learning 170
        
        3.2.1. Markov Decision Process 170
        
        3.2.3. SARSA 171
        
        3.2.2. Q-Learning 171
        
        3.3. States Estimation via DBNs 172
        
        3.3.1. Deep Belief Networks 172
        
        3.4. Numerical Results 174
        
        3.4.1. Commercial to Residential Transfer 178
        
        3.4.2. Residential to Residential Transfer 179
        
        4. Conclusions 181
        
        References 182
        
        Further Reading 185
        
        Chapter 8: Compressive Sensing for Power System Data Analysis 187
        
        1. Introduction 187
        
        2. Mathematical Modeling of a Compressive Sensing-Sparse Recovery Problem 189
        
        2.1. Compressive Sensing 189
        
        2.2. Sparse Recovery Problem 190
        
        3. Applications of CS-SR Techniques in Smart Grids 192
        
        4. Sparse Recovery-Based DSSE in Smart Grid 193
        
        5. Sparse Recovery-Based Fault Location in Smart Distribution and Transmission Networks 196
        
        6. Compressive Sensing-Based PD Pattern Recognition 199
        
        7. Conclusions and Future of the CS-SR in Smart Grids 203
        
        References 204
        
        Chapter 9: Time-Series Classification Methods: Review and Applications to Power Systems Data 207
        
        1. Introduction 207
        
        1.1. Contribution 208
        
        1.2. Notation 209
        
        2. The Classification Problem 212
        
        2.1. Classification Methods Taxonomy 213
        
        2.2. Computational Issues 214
        
        3. Data Sources 216
        
        3.1. Dimensionality Reduction 217
        
        3.1.1. DR Techniques Review 218
        
        3.2. Numerosity Reduction 218
        
        3.2.1. NR Techniques Review 218
        
        4. Classification Methods 220
        
        4.1. Feature-Based Methods 220
        
        4.1.1. Metrics-Based Approaches 221
        
        4.1.2. Occurrence Counting Approaches 223
        
        4.1.3. Dynamics-Based Approaches 225
        
        4.2. Distance-Based Methods 227
        
        4.2.1. Purely Distance-Based Methods 227
        
        4.2.2. Reduction Distance-Based Methods 230
        
        4.2.3. Dictionary Learning 231
        
        4.2.4. Parametric Distance-Based Methods 233
        
        4.3. Methods Comparison 236
        
        5. Applications 238
        
        6. Concluding Remarks 240
        
        References 241
        
        Section 3: Put the Power of Big Data into Power Systems 249
        
        Chapter 10: Future Trends for Big Data Application in Power Systems 251
        
        1. Introduction 251
        
        2. Transmission System 253
        
        2.1. Dynamic Behavior Analysis 254
        
        2.2. Steady-State Analysis 256
        
        2.3. TSO-DSO Cooperation 257
        
        3. Distribution System 258
        
        3.1. Monitoring and Situational Awareness 258
        
        3.2. Predictive Control and Management 260
        
        4. Asset Management 261
        
        5. Electricity Markets 262
        
        6. Demand-Side Flexibility 264
        
        7. Conclusions and Future Challenges 265
        
        References 266
        
        Acknowledgments 266
        
        Chapter 11: On Data-Driven Approaches for Demand Response 271
        
        1. Introduction 271
        
        2. Sources of Big Data in DR 273
        
        3. Big Data Applications in DR 273
        
        3.1. Assessment of Energy Consumption Behavior 276
        
        3.2. Electric Load Classification 277
        
        3.3. Demand and Renewable Energy Generation Forecasting 278
        
        3.4. Dynamic Pricing 281
        
        4. Real-World Applications and Research on Big Data-Driven Demand Response 283
        
        5. Summary and Future Prospects 285
        
        Acknowledgment 286
        
        References 286
        
        Chapter 12: Topology Learning in Radial Distribution Grids 289
        
        1. Introduction 289
        
        1.2. Technical Contribution 291
        
        1.1. Prior Work 291
        
        2. Distribution Grid: Structure and Power Flows 292
        
        Radial Structure 292
        
        PF Model 293
        
        3. Properties of Voltage Magnitudes in Radial Grids 294
        
        4. Topology Learning With Full Observation 298
        
        Algorithm Complexity 299
        
        Extension to Multiple Trees 299
        
        5. Topology Learning With Missing Data 299
        
        5.1. Missing Nodes Separated by Three or More Hops 300
        
        Complexity 301
        
        Note 302
        
        5.2. All Nonleaf Nodes Are Missing 302
        
        Computational Complexity 302
        
        6. Experiments 303
        
        7. Conclusions 305
        
        References 306
        
        Chapter 13: Grid Topology Identification via Distributed Statistical Hypothesis Testing 309
        
        1. Introduction 309
        
        1.1. Related Works 310
        
        2. Power Distribution Grid Model 311
        
        3. Voltage Correlation Analysis 313
        
        4. A Distributed Topology Test 317
        
        5. Numerical Experiments 322
        
        6. Conclusions 327
        
        References 327
        
        Chapter 14: Supervised Learning-Based Fault Location in Power Grids 331
        
        1. Fundamentals of SVM 332
        
        2. Power System Applications of SVM 334
        
        3. Fault Classification and Location for Three-Terminal Transmission Lines 336
        
        3.1. SVM-Based Fault Classification 337
        
        3.2. Single-Ended Traveling Wave-Based Fault Location 338
        
        3.3. Results and Discussion 340
        
        4. Fault Location for Hybrid HVAC Transmission Lines 343
        
        4.1. Single-Ended Traveling Wave-Based Fault Location 344
        
        4.2. Results and Discussion 345
        
        5. Summary 346
        
        References 347
        
        Chapter 15: Data-Driven Voltage Unbalance Analysis in Power Distribution Networks 349
        
        1. Introduction 349
        
        Chapter Points 349
        
        2. Problem Statement 350
        
        2.2. Utilized Voltage Band 351
        
        2.1. Unbalance in Low-Voltage Distribution Networks 351
        
        3. Data Acquisition and Storage 352
        
        3.1. Smart Meter Data Acquisition System 352
        
        3.2. Distributed Database Storage 353
        
        4. Distributed Data Processing 353
        
        4.1. Statistical Method 353
        
        4.2. Distributed Queries and Functions 355
        
        5. Data Discovery 355
        
        5.1. Distribution of Voltages 355
        
        5.2. Relation Between Meters With High Unbalance Events 355
        
        5.3. Timely Distribution of High Unbalance Events 358
        
        5.4. Relation Between Different Network States 361
        
        5.5. Maximum and Minimum Voltage: Voltage Spread 365
        
        6. Performance Evaluations 366
        
        6.1. Comparison Requirements 366
        
        6.1.1. Code 366
        
        6.1.4. System 367
        
        6.1.2. Data 367
        
        6.1.3. Database 367
        
        6.4. MapReduce Function MeterMinMax 368
        
        6.2. Evaluation Setup 368
        
        6.3. MapReduce Function CalcEventsLongFormat 368
        
        7. Conclusion 369
        
        References 369
        
        Chapter 16: Predictive Analytics for Comprehensive Energy Systems State Estimation 371
        
        1. Introduction 371
        
        2. Resource Forecasting 372
        
        2.1. Renewable Forecasting 372
        
        2.2. Wind Forecasting 373
        
        2.2.1. Wind Forecasting Overview 373
        
        2.2.2. Big Data-Driven Wind Forecasting 373
        
        Hours- to Day-Ahead NWP-Based Wind Forecasting 374
        
        Minutes- to 2-Hour-Ahead Machine-Learning-Based Wind Forecasting 375
        
        2.2.3. Wind Forecasting Datasets 377
        
        2.3. Solar Forecasting 378
        
        2.3.1. Solar Forecasting Overview 378
        
        2.3.2. Big Data-Driven Solar Forecasting 379
        
        Hours- to Day-Ahead NWP-Based Solar Forecasting 379
        
        Minutes- to 2-Hour-Ahead Sky Imaging-Based Solar Forecasting 380
        
        2.4. Renewable Forecasting Performance Evaluation Metrics 380
        
        3. User Energy System State Estimation 381
        
        3.1. Overview 381
        
        3.2. Load Forecasting 382
        
        3.2.1. Conventional Methods 382
        
        3.2.2. Artificial Intelligence-Based Methods 383

Pokaż więcej

Opis

Spis treści

Recenzje (0)