CNN?????????????????? CNN ?????

????????????????????? CNN ????????? CNN ???????????????????????? CNN ??????

——————-

???????????????????????????????????????????????????????????????????????1998?????18????????????????

?????????????????????????????????????????????????????????????? resNet?GoogleNet?Inception ????????????????????????????????????????????????????????????????????????????????????????????????

?????????????????????????AlexNet?LeNet?GoogLeNet?VGG-16?NiN?

????????????????????????????????????An Analysis of Deep Neural Network Models for Practical Applications?

LeNet5

1998??LeCun ???????LeNet?????????????????????1998???????????????????????????????????????????????????????????? LeNet ?? LeNet5???? Yann LeCun ???????????????????????5??????? lenet5??!

lenet5?????????????????????????????????????????????????????????????????????1998????? GPU ???????? CPU ???????????????????????????????????Lenet5??????????????????lenet5??????????????????????????????????????????????????????????????????? CNN ???????1. ?????2. ????3. ?????

LeNet5???

????????3??????????????????

????????????

?????????????

??????? tanh ?? sigmoid ??

??????(MLP)????????

?????????????????????

???? LeNet5????????????????????2010???????????????????? LeNet ??????

GAP

?1998?2010??10????????????????????????????????????2003???????2??????? SVM ????????????????20??????2012? coresea ????????????? SVM ????????????????????2????????????????????????????????? CPU ? GPU ???????????????????????????????????????????????2010????????????

Dan Ciresan Net

2010? Dan Claudiu Ciresan ? Jurgen Schmidhuber ????? GPU ?????????????? NVIDIA GTX280GPU ????????9???????

?????Nvidia ???????????????????????????

AlexNet

2012??Alex Krizhevsky ??? AlexNet???? LeNet ??????????? LeNet ?5?? AlexNet ?7??????? AlexNet ????2012?? ImageNet ??????AlexNet ??? LeNet ?????????????????????????????

AlexNet ???

?? ReLU ???????????? Sigmoid ???????

?? dropout ???????????????????????????

?? max-pooling ??

??? GPU NVIDIA GTX580????????

?? GPU ?????????????????10x ?????????????????????????????????????????AlexNet ?????????????????? CNN ???????????????? “?????????????”?

Network-in-network

2013????Min Lin ????????????1×1???????????????Network-in-network???????NiN ????????????????????????????????????? NiN ????? LeNet ?????(???????????)???????????????????????????????????

?? Min Lin ? NiN ???????? “?????”(NIN)???? CNN ???????????? NiN ??????????3×3256[conv] -> [maxpooling]????? NiN ??????????3×3256[conv] ->1×1256[conv] -> [maxpooling]?

MLP ?????????????????????????????????????????? ResNet ? Inception ????

VGG

2014?????????????????VGG?GoogLeNet?

???????VGG?????????????????3×3???????????????????????????????????????????????3×3????????3×3??????????????????????

???????????????????????????????????????????????VGG ??????????? LeNet ??????????? LeNet ????????????????????(????)?AlexNet ??????????????9×9?11×11????????????????????1×1??????? VGG ?????????????3×3???????????????????????????????????????? GoogleNet ? ResNet ????

???????????VGG ????3×3??????????????????????????????????????????????????3×3??????9????????7*7?????????????49???????????????????????????????????????????????????????????????

VGG ???????????????????????????????????????????????????????????????????????????

GoogLeNet

2014??? google ??? Christian Szegedy ????????????????????????????????GoogleNet?(??? Inception V1)?

?2014?????????????????????????????????????????????????????????????????????????????????????????????????????????????????? google ????????????????????????????????????

Christian ????????????????????????????????????????????????????????????????????????????????????? Christian ??? team ? google ????????

????????1×1,3×3?5×5????????????????????1×1???(NiN)??????????????????????? “bottleneck layer”?

Bottleneck layer

? NiN ????googleNet ? Bottleneck layer ??????????????????????????????????????????????????????????4???????????????????????????

??????????? Bottleneck layer??????256? feature map ???256? feature map ????? Inception ????3×3???????????? (256×256) x (3×3) ?????(??589,000?????)??????589,000?????? google ????????0.5ms ???????????????? Bottleneck layer ????????????????????256->641×1???????? Bottleneck layer ?????64??? feature map ????????64->2561×1??? ?????

256×64×1×1=16,000s

64×64×3×3=36,000s

64×256×1×1=16,000s

???70,000????????600,000?????10????!

?????????????????????????????Bottleneck layer ??? ImageNet ??????????????????????? ResNet ?????

?????????????????????????1×1??????????????????????????????????????????????

Inception V3

Christian ?????????2015?2?????????????? googleNet ?????Batch-normalized??Batch-normalized ????????????????????????????????????????????? “??” ???????????????????????????????????????????????????????????????????????????

2015?12????????????? GoogLeNet(Inception V3) ???????????????????? GoogLeNet ???GoogLeNet ?????

???????????????????????????? pooling ????? feature maps

????????????????????????????

????????????????????????

???3×3??

??????????????

????????? GoogleNet ???? pooling ? softmax ??????????

ResNet

2015?12? ResNet?????????? Inception v3?????????? ResNet ?????????????????????????????????????????????????

??????????????????????????????2?????????????????????????????????????2?????????????????!??????????????????????????????????????????1000???????????? LeNet ??5??AlexNet ?????7??

??????1×1????????????1/4?????3×3???????????1×1???????????????????????????????256?????????256?????????? Bottleneck Layer ??????????????????????????????

ResNet ???????????7×7???????????? pooling ????????????? GoogleNet ????? pooling ????? softmax ??????????? VGG19??190????????34?? ResNet ???36?????

ResNet ????

ResNet ?????????????????

ResNet ???????????????????? RNN??????????????????????

SqueezeNet

2016?11?????????????????????SqueezeNet: AlexNet-level accuracy with50x fewer parameters and <0.5MB model size?????? SqueezeNet ??? AlexNet ???????????? Alex ????50?????????????????????????? SegNet ?? GoogleNet ???????????? SqueezeNet ????????????????????

Xception

Xception????? ResNet ? Inception V4???????????????? Inception ???

? Xception ????????? Xception ???????36?????? ResNet-34????????????? ResNet ???????? Inception V4??????

?here??here???? Xception ??????

??

????????????????????????AlexNet ?????????????????????????????? AlexNet ?????????????????????????????????????????????????????????????????????? GPU ???????????????????????????????????????????????????????????????????????????????????????????????????????????

????????????????????????????????????????????????????2-3???????????????????????????????????????????????????? NVIDIA Driver PX2?????????? Titan X ??7??

??

??????????????????? “????” ?????????????????????????????????????????????????

????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????? CNN ??????????????????????????? HOG ????? SIFT ???????

????????????????????????????????????????????????????????????????????????????????

??

chenzomi12.github.io

LeNet

Dan Ciresan Net

AlexNet

Network-in-network

VGG

GoogleNet

Inception V3

Batch-normalized Layer

ResNet

SqueezeNet

Xception

Neural Network Architectures