文章目录
1.导入库2.数据准备2.1 加载训练集2.2 加载测试集2.3 对类别做One-Hot编码2.4 对图片像素的0-255值做归一化,并减去均值3.搭建神经网络3.1 定义函数resnet_layer,返回值是经过resnet_layer计算的结果3.2 定义函数resnet_v1,返回值是模型对象3.3 定义函数resnet_v2,返回值是模型对象3.4 实例化模型对象3.5 多GPU并行训练3.6 打印模型架构信息4. 模型训练4.1 规划学习率(训练到后期时学习率需减小)4.2 模型训练时的参数设置4.3 使用图像增强的结果做模型训练5.模型评估5.1 加载训练好的模型5.2 计算训练集的准确率5.3 计算测试集的准确率6. 模型测试结果可视化6.1 随机选100张图可视化6.2 随机选取100张图片的同时,要求10个类别,每个类别取10张7.Keras中权重文件的读写7.1 使用load_model方法加载模型文件7.2 使用save_weights方法保存权重文件7.3 使用load_weights方法加载权重文件8. 图像分类:纯终端运行8.1 在code文件夹下新建image_server.py写入下面服务端代码8.2 在code文件夹下新建image_client.py写入下面客户端代码8.3 cmd文件双击启动9.图像分类:有web页面9.1 新建_11_web_image_server_2.py文件写入下面服务端代码:9.2 新建templates文件夹里_12_web_page.html文件9.3 cmd文件双击启动10.目标检测:纯终端运行10.1 新建_21_yolov3.py文件10.2 cmd文件双击启动1.导入库
1.
keras(resnet):/keras-team/keras/blob/master/examples/cifar10_resnet.py
2.
用到的cifar10数据集和模型权重链接:/s/1L4oZAPg_9B_YipPeihmBwQ 提取码:8jcw
5个data_batch文件,每个1万数据
# 加入下面2行,可以使py代码文件中的修改即时生效%load_ext autoreload%autoreload 2# 下行如果python2使用print,也得加上括号from __future__ import print_functionimport keras# 导入keras库的这4种层:全连接层Dense,2维卷积层Conv2D,批量归一化层BatchNormalization,激活层Activationfrom keras.layers import Dense, Conv2D, BatchNormalization, Activation# 导入keras库的这3种层:平均2维池化层AveratePooling2D, 输入层Input,激活层Activationfrom keras.layers import AveragePooling2D, Input, Flatten# 导入keras库的优化器:Adam优化器from keras.optimizers import Adam# 导入keras库的回传函数:模型检查点ModelCheckpoint, LearningRateSchedulerfrom keras.callbacks import ModelCheckpoint, LearningRateScheduler# 导入keras库的回传函数:学习率递减ReduceLROnPlateaufrom keras.callbacks import ReduceLROnPlateau# 导入keras库的图片处理函数:图片数据生成器from keras.preprocessing.image import ImageDataGenerator# 导入keras库的正则化函数:L2正则化from keras.regularizers import l2# 导入keras库的后端:backend中文叫做后端,取别名为Kfrom keras import backend as K# 导入keras库的模型函数:Modelfrom keras.models import Model# 导入keras库的数据集类:cifar10from keras.datasets import cifar10# 导入必需的常用库import numpy as npimport os
2.数据准备
2.1 加载训练集
官方代码没有指定路径:
from keras.datasets.cifar import load_batch #load_batch用于加载指定路径## 加载数据集cifar10里面的训练集def load_train_dataset(dirPath='../resources/cifar-10-batches-py/'):train_sample_quantity = 50000image_width = 32image_height = 32channel_quantity = 3train_X = np.zeros((train_sample_quantity, channel_quantity, image_width, image_height),dtype='uint8')train_y = np.zeros((train_sample_quantity, ),dtype='uint8') #下面循环写入相同大小零矩阵for i in range(1, 6):fileName = 'data_batch_%d' %i #看文件名格式filePath = os.path.join(dirPath, fileName)startIndex = (i - 1) * 10000endIndex = i * 10000 #用到了load_batch,训练集定义5万,循环5次,0-1万行,1万-2万行....train_X[startIndex:endIndex, :, :, :], train_y[startIndex:endIndex] = load_batch(filePath)print('train_X矩阵转置前:', train_X.shape)# 从官网上下载的数据集的4个维度为样本个数n、通道数c、宽度w、高度h# Keras基于Tensorflow,数据的维度顺序要求:样本个数n、宽度w、高度h、通道数c,所以使用np.transpose完成矩阵转置train_X = train_X.transpose(0, 2, 3, 1)print('train_X矩阵转置后:', train_X.shape)return train_X, train_ydirPath = '../resources/cifar-10-batches-py/'train_imageData, train_y = load_train_dataset()
2.2 加载测试集
# 加载数据集cifar10里面的测试集def load_test_dataset(dirPath='../resources/cifar-10-batches-py/'):fileName = 'test_batch'filePath = os.path.join(dirPath, fileName)test_X, test_y = load_batch(filePath)print('test_X矩阵转置前:', test_X.shape)test_X = test_X.transpose(0, 2, 3, 1)print('test_X矩阵转置后:', test_X.shape)return test_X, test_ydirPath = '../resources/cifar-10-batches-py/'test_imageData, test_y = load_test_dataset()
2.3 对类别做One-Hot编码
# 对类别ID做One-Hot编码from keras.utils import to_categoricalclass_quantity = 10train_Y = to_categorical(train_y, class_quantity)test_Y = to_categorical(test_y, class_quantity)
2.4 对图片像素的0-255值做归一化,并减去均值
train_X = train_imageData.astype('float32') / 255test_X = test_imageData.astype('float32') / 255pixel_mean = np.mean(train_X, axis=0)print('pixel_mean.shape:', pixel_mean.shape)train_X = train_X - pixel_meantest_X = test_X - pixel_mean
3.搭建神经网络
3.1 定义函数resnet_layer,返回值是经过resnet_layer计算的结果
def resnet_layer(inputs, #定义了一层resnet_layernum_filters=16,kernel_size=3,strides=1,activation='relu',batch_normalization=True,conv_first=True):conv = Conv2D(num_filters,kernel_size=kernel_size,strides=strides,padding='same',kernel_initializer='he_normal',kernel_regularizer=l2(1e-4))x = inputsif conv_first: #为resnet V1x = conv(x) #conv为一函数,相当于下图weightif batch_normalization:x = BatchNormalization()(x)if activation is not None:x = Activation(activation)(x)else: #为resnet V2if batch_normalization:x = BatchNormalization()(x) #BatchNormalization()实例化一个函数对象if activation is not None:x = Activation(activation)(x)x = conv(x)return x
3.2 定义函数resnet_v1,返回值是模型对象
def resnet_v1(input_shape, depth, num_classes=10):if (depth - 2) % 6 != 0:raise ValueError('depth should be 6n+2 (eg 20, 32, 44 in [a])')# Start model definition.num_filters = 16num_res_blocks = int((depth - 2) / 6)inputs = Input(shape=input_shape)x = resnet_layer(inputs=inputs)# Instantiate the stack of residual unitsfor stack in range(3):for res_block in range(num_res_blocks):strides = 1if stack > 0 and res_block == 0: # first layer but not first stackstrides = 2 # downsampley = resnet_layer(inputs=x,num_filters=num_filters,strides=strides)y = resnet_layer(inputs=y,num_filters=num_filters,activation=None)if stack > 0 and res_block == 0: # first layer but not first stack# linear projection residual shortcut connection to match# changed dimsx = resnet_layer(inputs=x,num_filters=num_filters,kernel_size=1,strides=strides,activation=None,batch_normalization=False)x = keras.layers.add([x, y])x = Activation('relu')(x)num_filters *= 2# Add classifier on top.# v1 does not use BN after last shortcut connection-ReLUx = AveragePooling2D(pool_size=8)(x)y = Flatten()(x)outputs = Dense(num_classes,activation='softmax',kernel_initializer='he_normal')(y)# Instantiate model.model = Model(inputs=inputs, outputs=outputs)return model
3.3 定义函数resnet_v2,返回值是模型对象
def resnet_v2(input_shape, depth, num_classes=10):if (depth - 2) % 9 != 0:#深度必须是9n+2,比如20层,56层,110层raise ValueError('depth should be 9n+2 (eg 56 or 110 in [b])')# Start model definition.num_filters_in = 16 # 卷积核数量num_res_blocks = int((depth - 2) / 9)inputs = Input(shape=input_shape)x = resnet_layer(inputs=inputs, # resnet lay 0如下表,第一次调用了一层resnet_layernum_filters=num_filters_in,conv_first=True)# Instantiate the stack of residual units 实例化剩余单元的堆栈for stage in range(3):for res_block in range(num_res_blocks):activation = 'relu'batch_normalization = Truestrides = 1if stage == 0: #如果stage和res_block == 0,不进行activation,batch_normalization,num_filters_out = num_filters_in * 4if res_block == 0: # first layer and first stageactivation = Nonebatch_normalization = Falseelse:num_filters_out = num_filters_in * 2if res_block == 0: # first layer but not first stagestrides = 2 # downsample# bottleneck residual unity = resnet_layer(inputs=x,num_filters=num_filters_in,kernel_size=1,strides=strides,activation=activation,batch_normalization=batch_normalization,conv_first=False)y = resnet_layer(inputs=y,num_filters=num_filters_in,conv_first=False)y = resnet_layer(inputs=y,num_filters=num_filters_out,kernel_size=1,conv_first=False)if res_block == 0:# linear projection residual shortcut connection to match# changed dimsx = resnet_layer(inputs=x,num_filters=num_filters_out,kernel_size=1,strides=strides,activation=None,batch_normalization=False)x = keras.layers.add([x, y]) #实现shotcutnum_filters_in = num_filters_out#如上三个resnet_layer调用一个shotcut# Add classifier on top.# v2 has BN-ReLU before Poolingx = BatchNormalization()(x) #如下三行代码对应下表对应后面几行x = Activation('relu')(x)x = AveragePooling2D(pool_size=8)(x)y = Flatten()(x)outputs = Dense(num_classes,activation='softmax',kernel_initializer='he_normal')(y)# Instantiate model.model = Model(inputs=inputs, outputs=outputs)return model
3.4 实例化模型对象
# Model version# Orig paper: version = 1 (ResNet v1), Improved ResNet: version = 2 (ResNet v2)version = 2# Computed depth from supplied model parameter nn = 2if version == 1:depth = n * 6 + 2elif version == 2:depth = n * 9 + 2# 根据ResNet版本,获取对应的模型对象if version == 2:model = resnet_v2(input_shape=input_shape, depth=depth)else:model = resnet_v1(input_shape=input_shape, depth=depth)
3.5 多GPU并行训练
/matterport/Mask_RCNN/blob/master/mrcnn/parallel_model.py
打开parallel_model.py文件,在原来文件加入红点一行:
from parallel_model import ParallelModelgpu_count = 2model = ParallelModel(model, gpu_count)pile(loss='categorical_crossentropy',optimizer=Adam(lr=0.001),metrics=['accuracy'])
3.6 打印模型架构信息
# Model name, depth and versionmodel_type = 'ResNet%dv%d' % (depth, version)print(model_type)model.summary()
如下所示三个resnet_layer为一个shotcut,但如上图是两个layer再加,作者用的三个效果更好
用于数据集cifar10的图像分类网络ResNet20v2架构处理层_0: 输入层,调用keras.layers中的Input类,通过传递参数shape实例化对象数据层_0:矩阵形状: N x 32 x 32 x 3resnet_layer_0 #对应2.3节第一个x = resnet_layer #第一个3表示上层通道数或卷积核fliter数 处理层_1:conv2d_1卷积核形状: 3 x 3 x 3 x 16 / 1 #输入3大小3*3输出16,步长1,因为输出是16,所以加16个b #所以3*3*3*16=432(卷积核占的参数量)再加16为第一层conv2d_1参数个数448 数据层_1:矩阵形状: N x 32 x 32 x 16 #None表示batch size 处理层_2:batch_normalization_1数据层_2: 矩阵形状: N x 32 x 32 x 16处理层_3:activation_1数据层_3:矩阵形状: N x 32 x 32 x 16 resnet_layer_1 处理层_4:conv2d_2卷积核形状: 16 x 1 x 1 x 16 / 1 数据层_4:矩阵形状: N x 32 x 32 x 16resnet_layer_2 处理层_5:batch_normalization_2数据层_5:矩阵形状: N x 32 x 32 x 16处理层_6:activation_2数据层_6: 矩阵形状: N x 32 x 32 x 16处理层_7:conv2d_3卷积核形状: 16 x 3 x 3 x 16 / 1数据层_7:矩阵形状: N x 32 x 32 x 16resnet_layer_3 处理层_8: batch_normalization_3数据层_8:矩阵形状:N x 32 x 32 x 16处理层_9:activation_3 数据层_9:矩阵形状:N x 32 x 32 x 16处理层_10:conv2d_4卷积核形状: 16 x 1 x 1 x 64 / 1数据层_10:矩阵形状: N x 32 x 32 x 64shortcut_1 #不默认对上层处理,取指定层处理,相加,route_1是自己定义的处理层_11:route_1 取数据层_3 #数据层_3和下行数据层_11是同一层数据层_11:矩阵形状: N x 32 x 32 x 16 处理层_12:conv2d_5卷积核形状: 16 x 1 x 1 x 64 / 1#(32-1+0)/1 +1=32,公式里第一个32表示上层长32,第二个32是得出这层长#N x 32 x 32 x 16经过16 x 1 x 1 x 64 / 1处理得出N x 32 x 32 x 64即下行数据层_12:矩阵形状: N x 32 x 32 x 64 处理层_13:add_1把数据层_10、数据层_12相加数据层_13:矩阵形状: N x 32 x 32 x 64 /////////////////////////////////////////////////////////////////////// resnet_layer_4处理层_14:batch_normalization_4数据层_14:矩阵形状: N x 32 x 32 x 64处理层_15:activation_4数据层_15:矩阵形状: N x 32 x 32 x 64处理层_16:conv2d_6 卷积核形状: 64 x 1 x 1 x 16 / 1 数据层_16:矩阵形状: N x 32 x 32 x 16resnet_layer_5 处理层_17:batch_normalization_5数据层_17:矩阵形状: N x 32 x 32 x 16处理层_18:activation_5数据层_18:矩阵形状: N x 32 x 32 x 16处理层_19:conv2d_7 卷积核形状: 16 x 3 x 3 x 16 / 1 数据层_19:矩阵形状: N x 32 x 32 x 16resnet_layer_6处理层_20:batch_normalization_6数据层_20:矩阵形状: N x 32 x 32 x 16处理层_21:activation_6数据层_21:矩阵形状: N x 32 x 32 x 16处理层_22:conv2d_8 卷积核形状: 16 x 1 x 1 x 64 / 1数据层_22:矩阵形状: N x 32 x 32 x 64shortcut_2处理层_23:add_2把数据层_22、数据层_13相加数据层_23:矩阵形状: N x 32 x 32 x 64//////////////////////////////////////////////////////////////////////resnet_layer_7 处理层_24:batch_normalization_7 数据层_24:矩阵形状: N x 32 x 32 x 64处理层_25:activation_7数据层_25:矩阵形状: N x 32 x 32 x 64处理层_26:conv2d_9卷积核形状:64 x 1 x 1 x 64 / 2数据层_26:矩阵形状: N x 16 x 16 x 64 #通道扩展指最后位64变大,#下采样指由32*32变为16*16resnet_layer_8处理层_25:batch_normalization_8数据层_25:矩阵形状: N x 16 x 16 x 64处理层_26:activation_8数据层_26:矩阵形状: N x 16 x 16 x 64处理层_27:conv2d_10 卷积核形状:64 x 3 x 3 x 64 / 1数据层_27:矩阵形状: N x 16 x 16 x 64resnet_layer_9处理层_28:batch_normalization_9数据层_28:矩阵形状: N x 16 x 16 x 64处理层_29:activation_9数据层_29:矩阵形状 N x 16 x 16 x 64处理层_30:conv2d_11 卷积核形状:64 x 1 x 1 x 128 / 1数据层_30:矩阵形状 N x 16 x 16 x 128shortcut_3处理层_31:route_2 取数据层_23数据层_31:矩阵形状: N x 32 x 32 x 64处理层_32:conv2d_12 卷积核形状:64 x 1 x 1 x 128 / 2数据层_32:矩阵形状: N x 16 x 16 x 128处理层_33:add_3 把数据层_32、数据层_30相加数据层_33:矩阵形状: N x 16 x 16 x 128 ////////////////////////////////////////////////////////////////////resnet_layer_10 处理层_34:batch_normalization_10数据层_34:矩阵形状: N x 16 x 16 x 128处理层_35:activation_10数据层_35:矩阵形状: N x 16 x 16 x 128处理层_36:conv2d_13 卷积核形状:128 x 1 x 1 x 64 / 1数据层_36:矩阵形状: N x 16 x 16 x 64resnet_layer_11处理层_37:batch_normalization_11数据层_37:矩阵形状: N x 16 x 16 x 64处理层_38:activation_11数据层_38:矩阵形状: N x 16 x 16 x 64处理层_39:conv2d_14 卷积核形状: 64 x 3 x 3 x 64 / 1数据层_39:矩阵形状: N x 16 x 16 x 64resnet_layer_12 处理层_40:batch_normalization_12数据层_40:矩阵形状: N x 16 x 16 x 64处理层_41:activation_12数据层_41:矩阵形状: N x 16 x 16 x 64处理层_42:conv2d_15 卷积核形状: 64 x 1 x 1 x 128 / 1数据层_42:矩阵形状: N x 16 x 16 x 128shortcut_4处理层_43:add_4 把数据层_42、数据层_33相加数据层_43:矩阵形状: N x 16 x 16 x 128/////////////////////////////////////////////////////////////////////////resnet_layer_13 处理层_44:batch_normalization_13数据层_44:矩阵形状: N x 16 x 16 x 128处理层_45:activation_13数据层_45:矩阵形状: N x 16 x 16 x 128处理层_46:conv2d_16 卷积核形状: 128 x 1 x 1 x 128 / 2数据层_46:矩阵形状: N x 8 x 8 x 128 resnet_layer_14 处理层_47:batch_normalization_14数据层_47:矩阵形状: N x 8 x 8 x 128处理层_48:activation_14数据层_48:矩阵形状: N x 8 x 8 x 128处理层_49:conv2d_17 卷积核形状:128 x 3 x 3 x 128 / 1数据层_49:矩阵形状: N x 8 x 8 x 128resnet_layer_15 处理层_50:batch_normalization_15数据层_50:矩阵形状: N x 8 x 8 x 128处理层_51:activation_15数据层_51:矩阵形状: N x 8 x 8 x 128处理层_52:conv2d_18 卷积核形状:128 x 1 x 1 x 256 / 1数据层_52:矩阵形状: N x 8 x 8 x 256shortcut_5处理层_53:route_3取数据层_43数据层_53:矩阵形状: N x 16 x 16 x 128 处理层_54:conv2d_19 卷积核形状:128 x 1 x 1 x 256 / 2数据层_54:矩阵形状: N x 8 x 8 x 256处理层_55:add_5 把数据层_54、数据层_52相加数据层_55:矩阵形状: N x 8 x 8 x 256 //////////////////////////////////////////////////////////////////////// resnet_layer_16 处理层_56:batch_normalization_16数据层_56:矩阵形状: N x 8 x 8 x 256 处理层_57:activation_16数据层_57:矩阵形状: N x 8 x 8 x 256 处理层_58:conv2d_20 卷积核形状:256 x 1 x 1 x 128 / 1数据层_58:矩阵形状: N x 8 x 8 x 128 resnet_layer_17处理层_59:batch_normalization_17数据层_59:矩阵形状: N x 8 x 8 x 128处理层_60:activation_17数据层_60:矩阵形状: N x 8 x 8 x 128 处理层_61:conv2d_21 卷积核形状:128 x 3 x 3 x 128 / 1数据层_61:矩阵形状: N x 8 x 8 x 128resnet_layer_18处理层_62:batch_normalization_18数据层_62:矩阵形状: N x 8 x 8 x 128处理层_63:activation_18数据层_63:矩阵形状: N x 8 x 8 x 128 处理层_64:conv2d_22 卷积核形状:128 x 1 x 1 x 256 / 1数据层_64:矩阵形状: N x 8 x 8 x 256shortcut_6 #18层每3次一共做了6次shortcut处理层_62:add_6 把数据层_64、数据层_55相加数据层_62:矩阵形状: N x 8 x 8 x 256/////////////////////////////////////////////////////////////////////classifier#如果是yoloV3就是yolo层了处理层_63:batch_normalization_19数据层_63:矩阵形状: N x 8 x 8 x 256处理层_64:activation_19数据层_64:矩阵形状: N x 8 x 8 x 256处理层_65:average_pooling2d_1数据层_65:矩阵形状: N x 1 x 1 x 256处理层_66:flatten_1数据层_66:矩阵形状: N x 256处理层_67:dense_1数据层_67:矩阵形状: N x 10
所以ResNet20_V2:这个20是conv共19层(layer0)+dense1层
4. 模型训练
4.1 规划学习率(训练到后期时学习率需减小)
def lr_schedule(epoch):lr = 1e-3if epoch > 180:lr *= 0.5e-3elif epoch > 160:lr *= 1e-3elif epoch > 120:lr *= 1e-2elif epoch > 80:lr *= 1e-1print('Learning rate: ', lr)return lr
4.2 模型训练时的参数设置
# Training parametersbatch_size = 64 # orig paper trained all networks with batch_size=128epochs = 200# Prepare model model saving directory.save_dir = os.path.abspath('../resources/saved_models')model_name = 'cifar10_%s_model.{epoch:03d}.h5' % model_typeif not os.path.isdir(save_dir):os.makedirs(save_dir)filepath = os.path.join(save_dir, model_name)# Prepare callbacks for model saving and for learning rate adjustment.checkpoint = ModelCheckpoint(filepath=filepath,monitor='val_acc',verbose=0,save_best_only=True)lr_scheduler = LearningRateScheduler(lr_schedule)lr_reducer = ReduceLROnPlateau(factor=np.sqrt(0.1),cooldown=0,patience=5,min_lr=0.5e-6)callbacks = [checkpoint, lr_reducer, lr_scheduler]
4.3 使用图像增强的结果做模型训练
data_augmentation = Trueif data_augmentation:print('Using real-time data augmentation.')# This will do preprocessing and realtime data augmentation:datagen = ImageDataGenerator(# set input mean to 0 over the datasetfeaturewise_center=False,# set each sample mean to 0samplewise_center=False,# divide inputs by std of datasetfeaturewise_std_normalization=False,# divide each input by its stdsamplewise_std_normalization=False,# apply ZCA whiteningzca_whitening=False,# epsilon for ZCA whiteningzca_epsilon=1e-06,# randomly rotate images in the range (deg 0 to 180)rotation_range=0,# randomly shift images horizontallywidth_shift_range=0.1,# randomly shift images verticallyheight_shift_range=0.1,# set range for random shearshear_range=0.,# set range for random zoomzoom_range=0.,# set range for random channel shiftschannel_shift_range=0.,# set mode for filling points outside the input boundariesfill_mode='nearest',# value used for fill_mode = "constant"cval=0.,# randomly flip imageshorizontal_flip=True,# randomly flip imagesvertical_flip=False,# set rescaling factor (applied before any other transformation)rescale=None,# set function that will be applied on each inputpreprocessing_function=None,# image data format, either "channels_first" or "channels_last"data_format=None,# fraction of images reserved for validation (strictly between 0 and 1)validation_split=0.0)datagen.fit(x_train)# Fit the model on the batches generated by datagen.flow().model.fit_generator(datagen.flow(x_train, y_train, batch_size=batch_size),validation_data=(x_test, y_test),epochs=epochs,verbose=1, workers=4,callbacks=callbacks)
附:
python的生成器与keras.preprocessing.image文件的ImageDataGenerator类
的关系:
问号可以找在哪个路径:
如下图打开image.py发现是继承image:
找image:
5.模型评估
5.1 加载训练好的模型
from keras.models import load_modelfrom keras.optimizers import Adammodel_filePath = '../resources/saved_models/cifar10_ResNet56v2_model.162.h5'model = load_model(model_filePath)pile(loss='categorical_crossentropy',optimizer=Adam(lr=0.001),metrics=['accuracy'])
5.2 计算训练集的准确率
scores = model.evaluate(train_X, train_Y, verbose=1, batch_size=1000)print('Test loss:%.6f' %scores[0])print('Test accuracy:%.6f' %scores[1])
5.3 计算测试集的准确率
scores = model.evaluate(test_X, test_Y, verbose=1, batch_size=1000)print('Test loss:%.6f' %scores[0])print('Test accuracy:%.6f' %scores[1])
6. 模型测试结果可视化
6.1 随机选100张图可视化
import mathimport matplotlib.pyplot as pltimport numpy as np%matplotlib inlineimport matplotlibmatplotlib.rcParams['font.sans-serif'] = ['SimHei']import randomdef draw_image(position, image, title, isTrue):plt.subplot(*position)plt.imshow(image)plt.axis('off')if not isTrue:plt.title(title, color='red')else:plt.title(title)def batch_draw_images(model, batch_size, test_imageData, test_X, test_y, id2name_dict):index_list = list(range(len(test_imageData)))selected_index_list = random.sample(index_list, batch_size)true_imageData = test_imageData[selected_index_list]true_X = test_X[selected_index_list]true_y = np.array(test_y)[selected_index_list]predict_Y = model.predict(true_X)predict_y = np.argmax(predict_Y, axis=1)row_number = math.ceil(batch_size ** 0.5)column_number = row_numberplt.figure(figsize=(row_number+8, column_number+8))for i in range(row_number):for j in range(column_number):index = i * column_number + jif index < batch_size:position = (row_number, column_number, index+1)image = true_imageData[index]actual_classId = true_y[index]predict_classId = predict_y[index]isTrue = actual_classId==predict_classIdactual_className = id2name_dict[actual_classId]predict_className = id2name_dict[predict_classId]title = 'actual:%s\npredict:%s' %(actual_className, predict_className)draw_image(position, image, title, isTrue)batch_size = 100 #展示100张图className_list = ['飞机', '汽车', '鸟', '猫', '鹿', '狗', '青蛙', '马', '船', '卡车']id2name_dict = {a:b for a, b in enumerate(className_list)}batch_draw_images(model, batch_size, test_imageData, test_X, test_y, id2name_dict)plt.show()
6.2 随机选取100张图片的同时,要求10个类别,每个类别取10张
def get_selectedIndexList(test_y, batch_size):assert batch_size % 10 == 0, 'batch_size must be times by 10, or you change function get_selectedIndexList'column_number = int(batch_size / 10)classId_ndarray = np.unique(test_y)selected_index_list = []for i, classId in enumerate(classId_ndarray):index_ndarray = np.where(test_y==classId)[0]selected_index_ndarray = np.random.choice(index_ndarray, column_number)selected_index_list.extend(selected_index_ndarray.tolist())return selected_index_list def batch_draw_images_2(model, selected_index_list, test_imageData, test_X, test_y, id2name_dict):true_imageData = test_imageData[selected_index_list]true_X = test_X[selected_index_list]true_y = np.array(test_y)[selected_index_list]predict_Y = model.predict(true_X)predict_y = np.argmax(predict_Y, axis=1)row_number = math.ceil(batch_size ** 0.5)column_number = row_numberplt.figure(figsize=(row_number+8, column_number+8))for i in range(row_number):for j in range(column_number):index = i * column_number + jif index < batch_size:position = (row_number, column_number, index+1)image = true_imageData[index]actual_classId = true_y[index]predict_classId = predict_y[index]isTrue = actual_classId==predict_classIdactual_className = id2name_dict[actual_classId]predict_className = id2name_dict[predict_classId]title = 'actual:%s\npredict:%s' %(actual_className, predict_className)draw_image(position, image, title, isTrue)batch_size = 100className_list = ['飞机', '汽车', '鸟', '猫', '鹿', '狗', '青蛙', '马', '船', '卡车']id2name_dict = {a:b for a, b in enumerate(className_list)}selected_index_list = get_selectedIndexList(test_y, batch_size)batch_draw_images_2(model, selected_index_list, test_imageData, test_X, test_y, id2name_dict)plt.show()
7.Keras中权重文件的读写
7.1 使用load_model方法加载模型文件
from keras.models import load_modelfrom keras.optimizers import Adammodel_filePath = '../resources/saved_models/cifar10_ResNet56v2_model.162.h5'model = load_model(model_filePath)pile(loss='categorical_crossentropy',optimizer=Adam(lr=0.001),metrics=['accuracy'])
7.2 使用save_weights方法保存权重文件
weights_h5FilePath = '../resources/saved_models/resnet56v2_weights.h5'model.save_weights(weights_h5FilePath)
7.3 使用load_weights方法加载权重文件
input_shape = (32, 32, 3)depth = 56model = resnet_v2(input_shape, depth)weights_h5FilePath = '../resources/saved_models/resnet56v2_weights.h5'model.load_weights(weights_h5FilePath)pile(loss='categorical_crossentropy',optimizer=Adam(lr=0.001),metrics=['accuracy'])
8. 图像分类:纯终端运行
服务端
开启服务后,flask库等待接收post请求
。
客户端
在cmd中传入需要检测图片的路径,requests库发起post请求
并打印返回值。
新建:模型部署(在线预测)文件夹,创建以下两个文件夹,将网盘资源放入resources中。链接:/s/1X0PbGLnjO2vQsQFcwdMgYw 提取码:z8ax
在resources文件夹中:
在yolov3文件夹中:
8.1 在code文件夹下新建image_server.py写入下面服务端代码
# -*- coding: utf-8 -*-# 导入常用的库import timeimport osimport cv2import numpy as npfrom flask import request, Flask # 导入flask库的Flask类和request对象app = Flask(__name__) #实例化Flask对象,这个对象变量是appimport pickle # 训练时保存的pixel_mean.pickle ,导入pickle加载图像数据要减去的(像素均值pixel_mean)with open('../resources/pixel_mean.pickle', 'rb') as file: pixel_mean = pickle.load(file) #'rb'是以二进制流读入文件赋给file变量className_list = ['plane', 'car', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']id2name_dict = {a:b for a, b in enumerate(className_list)} # 定义字典id2name_dict:把种类索引 转换为 种类名称 from keras.models import load_modelfrom keras.optimizers import Adammodel_filePath = '../resources/cifar10_ResNet56v2_model.162.h5' # 加载图像分类模型ResNet56model = load_model(model_filePath)pile(loss='categorical_crossentropy',optimizer=Adam(lr=0.001),metrics=['accuracy']) def get_imageNdarray(imageFilePath):image_ndarray = cv2.imread(imageFilePath) # 根据图片文件路径,cv2.imread获取图像数据矩阵 resized_image_ndarray = cv2.resize(image_ndarray,(32, 32), #大小统一变为32*32interpolation=cv2.INTER_AREA)return resized_image_ndarraydef process_imageNdarray(image_ndarray, pixel_mean): #模型预测前必要的图像处理,image_ndarray是32*32*3和pixel_mean大小一样rgb_image_ndarray = image_ndarray[:, :, ::-1]image_ndarray_1 = rgb_image_ndarray / 255image_ndarray_2 = image_ndarray_1 - pixel_meanreturn image_ndarray_2def predict_image(model, imageFilePath, id2name_dict): #使用模型对指定图片文件路径完成图像分类,返回值为预测的种类名称 image_ndarray = get_imageNdarray(imageFilePath) #get_imageNdarray方法获得图像数据矩阵processed_image_ndarray = process_imageNdarray(image_ndarray, pixel_mean)#process_imageNdarray方法获得图像处理后的数据inputs = processed_image_ndarray[np.newaxis, ...] #np.newaxis将32*32*3变为1*32*32*3,因为模型只接受4维矩阵predict_Y = model.predict(inputs)[0] # 4维第一维表示batch,所以可输入多张图片predict_y = np.argmax(predict_Y) # Y:对于每个类别的预测概率,y:哪个类别概率最大predict_className = id2name_dict[predict_y] # 种类id转成种类名称print('对此图片路径 %s 的预测结果为 %s' %(imageFilePath, predict_className))return predict_className@app.route("/", methods=['POST'])# 只有下面函数和服务端相关,其他都是定义一个函数然后返回结果# 定义回调(callback)函数,接收来自/的post请求,并返回预测结果。回调函数就像是开了一个线程,我先运行主线程,相关的计算问题抛给回调函数,让它先算,我继续运行我的程序,等你算出来了发回给我就行,不影响我当前执行def anyname_you_like(): # 其他地方不会引用该函数startTime = time.time() # 单位是秒received_file = request.files['file'] #往下涉及到了前端:因为request(从flask库导入)是个对象所以才有.files属性,里面的'file'对应image_client.py中file_dict(按要求输入)中'file'imageFileName = received_file.filename #received_file类似于列表if received_file: # 如果不为空received_dirPath = '../resources/received_images' #上一级文件夹if not os.path.isdir(received_dirPath):os.makedirs(received_dirPath) #如果没有的话就创建它imageFilePath = os.path.join(received_dirPath, imageFileName)#(图像文件名)放入(图像文件夹)中形成(图像文件路径)received_file.save(imageFilePath)#将(接收到的文件)保存到(文件路径)print('图片文件保存到此路径:%s' % imageFilePath)usedTime = time.time() - startTimeprint('接收图片并保存,总共耗时%.2f秒' % usedTime)startTime = time.time() #调用predict_image方法,上面定义过predict_className = predict_image(model, imageFilePath, id2name_dict)usedTime = time.time() - startTimeprint('完成对接收图片的分类预测,总共耗时%.2f秒' % usedTime)return predict_classNameelse:return 'failed'# 主函数 if __name__ == "__main__":print('在开启服务前,先测试predict_image函数')imageFilePath = '../resources/images/001.jpg'predict_className = predict_image(model, imageFilePath, id2name_dict) #测试函数有没有写对,写错的话下一步就不进行,model是在外面定义的,在main方法里可以用。app.run("127.0.0.1", port=5000) #第一个参数ip地址,第二个参数是开在哪个端口#如果接收到 127.0.0.1:5000/ 的话调用上面回调函数anyname_you_like()方法,如果/后加其他东西就要写其他回调函数
8.2 在code文件夹下新建image_client.py写入下面客户端代码
import requestsimport osif __name__ == "__main__": # 主函数 url = "http://127.0.0.1:5000"while True: #可循环输入图片input_content = input('输入图片路径,输入-1退出,默认值(../resources/images/001.jpg): ') #input()以字符串的方式获取用户输入if input_content.strip() == "": input_content = '../resources/images/001.jpg'if input_content.strip() == "-1":breakelif not os.path.exists(input_content.strip()):print('输入图片路径不正确,请重新输入')else:imageFilePath = input_content.strip() # 下面是正文:怎么把数据发过去# 下面一行用os.path.split将文件路径中文件名提取出来,第一段文件夹名,第二段文件名imageFileName = os.path.split(imageFilePath)[1] # 取第二段文件名file_dict = {# file_dict为一字典,键是'file',值是一元组包含三元素 #必须按file_dict方法传入'file':(imageFileName, # 做post请求时有这三行规范,第一段文件名open(imageFilePath,'rb'), # 第二段二进制流'image/jpg')} # 第三段后缀名类型 result = requests.post(url, files=file_dict) # 这里files参数(默认)对应服务端的filespredict_className = result.text # result是响应response为一对象有.text属性,result.text为一字符串print('图片路径:%s 预测结果为:%s\n' %(imageFilePath, predict_className))
8.3 cmd文件双击启动
在code文件夹下新建名为开启服务_ResNet56v2图片分类模型.cmd
文件,写入:
python image_server.pypause
在code文件夹下新建名为客户端发送图片数据做分类检测.cmd
文件,写入:
python image_client.pypause
运行结果:传入如下图图片路径如:../resources/images/004jpg
9.图像分类:有web页面
提交post请求后,不做网页跳转,停留在原始网页。从而使图像、预测种类名同时显示在同一网页,在网页中可以改变图片的显示大小。
9.1 新建_11_web_image_server_2.py文件写入下面服务端代码:
# -*- coding: utf-8 -*-# 导入常用的库import timeimport osimport cv2import numpy as np# 导入flask库from flask import Flask, render_template, request, jsonify# render_template为返回一个网页,返回jsonify数据# Flask, render_template, request, jsonify是四个方法from keras.models import load_modelfrom keras.optimizers import Adam# 导入pickle库import pickle# 实例化Flask对象app = Flask(__name__)# 设置开启web服务后,如果更新html文件,可以使更新立即生效app.jinja_env.auto_reload = Trueapp.config['TEMPLATES_AUTO_RELOAD'] = True# 调用pickle库的load方法,加载图像数据处理时需要减去的像素均值pixel_meanwith open('../resources/pixel_mean.pickle', 'rb') as file:pixel_mean = pickle.load(file)# 加载已经训练好的cifar10数据集的ResNet56图片分类模型的模型文件model_filePath = '../resources/cifar10_ResNet56v2_model.162.h5'model = load_model(model_filePath)pile(loss='categorical_crossentropy',optimizer=Adam(lr=0.001),metrics=['accuracy']) # 定义函数classId_to_className,把种类索引转换为种类名称def classId_to_className(classId):category_list = ['plane', 'car', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']className = category_list[classId]return className# 根据图片文件的路径获取图像矩阵 def get_imageNdarray(imageFilePath):image_ndarray = cv2.imread(imageFilePath)resized_image_ndarray = cv2.resize(image_ndarray,(32, 32),interpolation=cv2.INTER_AREA)return resized_image_ndarray# 模型预测前必要的图像处理 def process_imageNdarray(image_ndarray, pixel_mean):rgb_image_ndarray = image_ndarray[:, :, ::-1]image_ndarray_1 = rgb_image_ndarray / 255image_ndarray_2 = image_ndarray_1 - pixel_meanreturn image_ndarray_2# 使用模型对指定图片文件路径完成图像分类,返回值为预测的种类名称 def predict_image(model, imageFilePath):image_ndarray = get_imageNdarray(imageFilePath)processed_image_ndarray = process_imageNdarray(image_ndarray, pixel_mean)inputs = processed_image_ndarray[np.newaxis, ...]predict_Y = model.predict(inputs)[0]predict_y = np.argmax(predict_Y)predict_classId = predict_ypredict_className = classId_to_className(predict_classId)print('对此图片路径 %s 的预测结果为 %s' %(imageFilePath, predict_className))return predict_className# 访问首页时的调用函数 @app.route('/')def index_page(): # flask库要求'_05_web_page.html'必须在templates文件夹下 return render_template('_05_web_page.html') # anyname_you_like()是使用predict_image这个API服务时的调用函数 ,POST指向一个url链接 # 下行/前:.route是IP地址和端口。/后:predict_image:调用下行anyname_you_like()方法@app.route("/predict_image", methods=['POST'])def anyname_you_like():startTime = time.time()# 解析接收到的图片 # post传送 request请求 ,post过来是个字典file相当于一个键received_file = request.files['input_image'] #input_image对应_05_web_page.html中input_imageimageFileName = received_file.filenameif received_file:# 保存接收的图片到指定文件夹received_dirPath = '../resources/received_images'if not os.path.isdir(received_dirPath):os.makedirs(received_dirPath)imageFilePath = os.path.join(received_dirPath, imageFileName)received_file.save(imageFilePath)print('图片文件保存到此路径:%s' % imageFilePath)usedTime = time.time() - startTimeprint('接收图片并保存,总共耗时%.2f秒' % usedTime)# 对指定图片路径的图片做分类预测,并打印耗时,返回预测种类名称startTime = time.time()predict_className = predict_image(model, imageFilePath)usedTime = time.time() - startTimeprint('完成对接收图片的分类预测,总共耗时%.2f秒' % usedTime)return jsonify(predict_className=predict_className) # jsonify是个字典# 所有post请求返回要么jsonify,要么新的网页链接# 主函数(和之前一样) if __name__ == "__main__":print('在开启服务前,先测试predict_image函数')imageFilePath = '../resources/images/001.jpg'predict_className = predict_image(model, imageFilePath)app.run("127.0.0.1", port=5000)
9.2 新建templates文件夹里_12_web_page.html文件
<!DOCTYPE html><html lang="en"><head><meta charset="UTF-8"> #meta参数意思,字符编码设为UTF-8<title>实现需求web</title> #对应页面标题<script src="/jquery-3.3.1.min.js"></script> #从网络下载并运用jQuery库<script>$(document).ready(function(){#$符就是find element$("#image_file").change(function(){#不管什么标签,只要id="image_file",就找到下面input这个元素,只要发生改变change,函数function()作为参数传入change这个方法里var file = $(this)[0].files[0]; #取到文件$(".img").attr("src", URL.createObjectURL(file));}); # #号表示id=,.号表示class=$("button#button_1").click(function(){#当被点击促发function()函数var formData = new FormData($("#upload_form")[0]);#找("#upload_form")这个元素,结果是一个list,找到第一个。$("#upload_form")[0]这个作为参数传给FormData,FormData变为对象再赋值给formData变量,formData变量在下面被调用,不能写在函数外,唯独python可以$.ajax({#字典类型url: "/predict_image", #对应.py文件中"/predict_image"发起POST请求type: 'POST',data: formData, #发送表格数据processData: false,contentType: false,success: function(return_data){#如果发出请求成功传回结果$("#image_className").text("图片中物体的种类名称:" + return_data["predict_className"]) #找到"#image_className"标签,改文本内容为后面}, #"predict_className"是个键,找到对应值error: function(return_data){#返回的状态码是404alert("上传失败!")}}) });}); </script></head><body> #<body>里是网页显示的内容<form id="upload_form" enctype="multipart/form-data"> #</form>做表格,方便做post请求<input type="file" name="input_image" id="image_file"/></form> <div> #<div>就是分开一行<img src="" class="img" /></div><button id="button_1">提交信息</button> #设置一个按钮显示提交信息<p id="image_className"></p> #p代表文本</body></html>
9.3 cmd文件双击启动
在code文件夹下新建名为_11_开启web服务.cmd
文件,写入
python _11_web_image_server_2.pypause
复制下图绿框到浏览器,每上传一张图,resources文件夹里多出received_images文件夹,里面存着上传的图片。
10.目标检测:纯终端运行
10.1 新建_21_yolov3.py文件
# -*- coding: utf-8 -*-# 导入常用的库import osimport timeimport numpy as npimport cv2# 导入keras库from keras import backend as Kfrom keras.layers import Input# 导入yolo3文件夹中mode.py、utils.py这2个代码文件中的方法from yolo3.model import yolo_eval, yolo_bodyfrom yolo3.utils import letterbox_image# 导入PIL画图库from PIL import Image, ImageFont, ImageDraw# 通过种类的数量,每个种类对应的颜色,颜色变量color为rgb这3个数值组成的元组import colorsysdef get_colorList(category_quantity):hsv_list = []for i in range(category_quantity):hue = i / category_quantitysaturation = 1value = 1hsv = (hue, saturation, value)hsv_list.append(hsv)colorFloat_list = [colorsys.hsv_to_rgb(*k) for k in hsv_list]color_list = [tuple([int(x * 255) for x in k]) for k in colorFloat_list]return color_list # 定义类Detectorclass Detector(object):defaults = {"weights_h5FilePath": '../resources/yolov3/yolov3_weights.h5',"anchor_txtFilePath": '../resources/yolov3/yolov3_anchors.txt',"category_txtFilePath": '../resources/yolov3/coco.names',"score" : 0.3,"iou" : 0.35,"model_image_size" : (416, 416) #must be a multiple of 32}@classmethoddef get_defaults(cls, n):if n in cls.defaults:return cls.defaults[n]else:return 'Unrecognized attribute name "%s"' %n# 实例化对象后的初始化方法def __init__(self, **kwargs):self.__dict__.update(self.defaults) # 设置默认值self.__dict__.update(kwargs) # 根据传入的参数更新self.category_list = self.get_categoryList()self.anchor_ndarray = self.get_anchorNdarray()self.session = K.get_session()self.boxes, self.scores, self.classes = self.generate()# 从文本文件中解析出物体种类列表category_list,要求每个种类占一行def get_categoryList(self):with open(self.category_txtFilePath, 'r', encoding='utf8') as file:fileContent = file.read()line_list = [k.strip() for k in fileContent.split('\n') if k.strip()!='']category_list = line_listreturn category_list # 从表示anchor的文本文件中解析出anchor_ndarraydef get_anchorNdarray(self):with open(self.anchor_txtFilePath, 'r', encoding='utf8') as file:number_list = [float(k) for k in file.read().split(',')]anchor_ndarray = np.array(number_list).reshape(-1, 2)return anchor_ndarray# 加载模型,准备好绘图的颜色,准备模型运算的输出def generate(self):# 在Keras中,如果模型训练完成后只保存了权重,那么需要先构建网络,再加载权重文件num_anchors = len(self.anchor_ndarray)num_classes = len(self.category_list)self.yolo_model = yolo_body(Input(shape=(None, None, 3)),num_anchors//3,num_classes)self.yolo_model.load_weights(self.weights_h5FilePath)# 给不同类别的物体准备不同颜色的方框category_quantity = len(self.category_list)self.color_list = get_colorList(category_quantity)# 目标检测的输出:方框box,得分score,类别classself.input_image_size = K.placeholder(shape=(2, ))boxes, scores, classes = yolo_eval(self.yolo_model.output,self.anchor_ndarray,category_quantity,self.input_image_size,score_threshold=self.score,iou_threshold=self.iou)return boxes, scores, classes# 检测图片def detect_image(self, image):startTime = time.time()# 模型网络结构运算所需的数据准备boxed_image = letterbox_image(image, tuple(reversed(self.model_image_size)))image_data = np.array(boxed_image).astype('float') / 255image_data = np.expand_dims(image_data, 0) # Add batch dimension.# 模型网络结构运算out_boxes, out_scores, out_classes = self.session.run([self.boxes, self.scores, self.classes],feed_dict={self.yolo_model.input: image_data,self.input_image_size: [image.size[1], image.size[0]],})# 调用ImageFont.truetype方法实例化画图字体对象font = ImageFont.truetype(font='../resources/yolov3/FiraMono-Medium.otf',size=np.floor(2e-2 * image.size[1] + 0.5).astype('int32'))thickness = (image.size[0] + image.size[1]) // 300# 循环绘制若干个方框for i, c in enumerate(out_classes):# 调用ImageDraw.Draw方法实例化画图对象draw = ImageDraw.Draw(image)# 定义方框上方文字内容predicted_class = self.category_list[c]score = out_scores[i]label = '{} {:.2f}'.format(predicted_class, score)label_size = draw.textsize(label, font)box = out_boxes[i]top, left, bottom, right = boxtop = max(0, np.floor(top + 0.5).astype('int32'))left = max(0, np.floor(left + 0.5).astype('int32'))bottom = min(image.size[1], np.floor(bottom + 0.5).astype('int32'))right = min(image.size[0], np.floor(right + 0.5).astype('int32'))# 如果方框在图片中的位置过于靠上,调整文字区域if top - label_size[1] >= 0:text_origin = np.array([left, top - label_size[1]])else:text_origin = np.array([left, top + 1])# 方框厚度为多少,则画多少个矩形for j in range(thickness):draw.rectangle([left + j, top + j, right - j, bottom - j],outline=self.color_list[c])# 绘制方框中的文字draw.rectangle([tuple(text_origin), tuple(text_origin + label_size)],fill=self.color_list[c])draw.text(text_origin, label, fill=(0, 0, 0), font=font)del draw# 打印检测图片使用的时间usedTime = time.time() - startTimeprint('检测这张图片用时%.2f秒' %(usedTime))return image# 使用cv2库显示图片def cv2_display(image_ndarray):windowName = "object_detection_result"cv2.imshow(windowName, image_ndarray)while True:pressKey = cv2.waitKey(0)# 按Esc键或者q键可以关闭显示窗口if 27 == pressKey or ord('q') == pressKey:cv2.destroyAllWindows()break# 主函数if __name__ == "__main__":detector = Detector(weights_h5FilePath='../resources/yolov3/yolov3_weights.h5',anchor_txtFilePath='../resources/yolov3/yolov3_anchors.txt',category_txtFilePath='../resources/yolov3/coco.names')while True:input_content = input('输入图片路径,输入-1退出,默认值(../resources/images/person.jpg): ')if input_content.strip() == "":input_content = '../resources/images/person.jpg'if input_content.strip() == "-1":breakelif not os.path.exists(input_content.strip()):print('输入图片路径不正确,请重新输入')else:imageFilePath = input_content.strip()image = Image.open(imageFilePath)drawed_image = detector.detect_image(image)# 使用cv2库显示已经画框的图片,cv2库图像数据矩阵第3维度的通道顺序是bgrdrawed_image_ndarray = np.array(drawed_image)bgr_image_ndarray = drawed_image_ndarray[..., ::-1]cv2_display(bgr_image_ndarray)
10.2 cmd文件双击启动
在code文件夹下新建名为_22_使用YOLOv3检测单张图片.cmd
文件,写入
python _21_yolov3.pypause
在cmd中输入图片路径,或者按Enter键使用默认值,运行结果如下图所示。在cv窗口界面可以按Esc键或者q键退出,在cmd中可以多次输入路径检测多张图片。