Neural Network

<Neural Network>

    1. Deep Learning

    2. Perceptron

    3. Neural Network Architecture

    4. Forward & BackPropagation

---

Deep Learning

 

 

 

Deep Learning

Image source:&nbsp;https://niessner.github.io/I2DL/

 

• Few feature engineering
• State-ot-the-art performance
• GPU 발달 → Deep Learning 소요시간 감소

 

 

Deep Learning with Big Data

→ Data수가 많아질수록 Deep Learning의 성능 향상

 

 

Deep Learning 활용 분야

Game AI, Face Detection & Recognition, Object Detection & Recognition, Image Captioning, Machine Translation ...

---

Perceptron

 

Single Layer Perceptron

Image source: Andrew Ng

 

AND Gate

 

Image source: Andrew Ng

 

OR Gate

Image source: Andrew Ng

 

 

Decision Boundary

Image source: Andrew Ng

 

XOR Gate

→ single layer perceptron으로는 XOR 문제 해결할 수 없다.

 

Image source: Andrew Ng

→ Linear한 Decision Boundary로는 해결할 수 없다.

→ 2 single layer perceptron 사용

 

 

 

Multi Layer Perceptron

2 single layer perceptron으로 XOR 문제 해결 가능

 

Image source: Andrew Ng

 

Image source: Andrew Ng

 

→ OR Gate perceptron과 AND Gate perceptron 합성하여 Multi Layer Perceptron 생성

 

 

 

Neural Network Architecture

 

2-layer Nueral Network (1 hidden + 1 output layer) 1-hidden-layer Neural Network

3-layer Nueral Network (2 hidden + 1 output layer) 2-hidden-layer Neural Network

 

---

Forward & Backpropagation

 

Image source: Andrew Ng

 

 

 

f(x, y, z) = (x + y)z

e.g. x = -2, y = 5, z = -4

Image source: Andrew Ng

 

q(= x + y)는 3으로, f(= q * z)는 -12로 Computation

 

 

Forward pass (Chain Rule 사용) → Local Gradient

Chain Rule 예시

df/dx = z = -4

df/dy = z = -4

df/dz = x + y = -6

 

 

Backpropagation (Chain Rule 사용) → Global Gradient

Image source: Andrew Ng

df/df = 1

df/dg = -4

 

df/dx = df/dg · dg/dx = (-4) · 1 = -4

df/dy = df/dg · dg/dy = (-4) · 1 = -4

df/dz = 3

 

• 아무리 깊은 layer라도 Chain Rule로 미분값을 얻을 수 있다.
• Forward pass시, Local Gradient를 미리 구하여 저장한다.
• 저장해둔 Local Gradient와 Global Gradient를 Backward Pass시 곱하여 최종 미분값을 구한다. (df/dx, df/dy, df/dz)