tf_numpyDataLoad

이번 글은 간단합니다.

Numpy 배열에서 데이터를 로드합니다.

데이터를 로드하고 모델을 만들어 훈련하고 검증하는 절차입니다.

머신 러닝의 기본 과정이 이와 비슷하다고 생각하면 될 것 같습니다.

 

데이터로드 -> 모델 빌더 -> 훈련 -> 예측

 

 

<참조 : https://www.tensorflow.org/tutorials/load_data/numpy?hl=ko>

In [ ]:

Numpy Data Load¶

01. 설정¶

In [1]:

import numpy as np
import tensorflow as tf

In [2]:

# .npz 파일에서 로드하기

DATA_URL = 'https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz'

path = tf.keras.utils.get_file('mnist.npz', DATA_URL)
with np.load(path) as data:
    train_examples = data['x_train']
    train_labels = data['y_train']
    test_examples = data['x_test']
    test_labels = data['y_test']

In [3]:

train_dataset = tf.data.Dataset.from_tensor_slices((train_examples, train_labels))
test_dataset = tf.data.Dataset.from_tensor_slices((test_examples, test_labels))

In [ ]:

02. 데이터세트 사용하기¶

02. 01. 데이터세트 셔플 및 배치¶

In [4]:

BATCH_SIZE = 64
SHUFFLE_BUFFER_SIZE = 100

train_dataset = train_dataset.shuffle(SHUFFLE_BUFFER_SIZE).batch(BATCH_SIZE)
test_dataset = test_dataset.batch(BATCH_SIZE)

In [ ]:

02. 02. 모델 빌드 및 훈련¶

In [5]:

model = tf.keras.Sequential([
    tf.keras.layers.Flatten(input_shape=(28, 28)),
    tf.keras.layers.Dense(128, activation='relu'),
    tf.keras.layers.Dense(10)
])

model.compile(optimizer=tf.keras.optimizers.RMSprop(),
             loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
             metrics=['sparse_categorical_accuracy'])

In [6]:

model.fit(train_dataset, epochs=10)

Epoch 1/10
938/938 [==============================] - 10s 9ms/step - loss: 3.1527 - sparse_categorical_accuracy: 0.8755
Epoch 2/10
938/938 [==============================] - 8s 9ms/step - loss: 0.4964 - sparse_categorical_accuracy: 0.9289
Epoch 3/10
938/938 [==============================] - 8s 9ms/step - loss: 0.3787 - sparse_categorical_accuracy: 0.9457
Epoch 4/10
938/938 [==============================] - 8s 9ms/step - loss: 0.3175 - sparse_categorical_accuracy: 0.9557
Epoch 5/10
938/938 [==============================] - 8s 9ms/step - loss: 0.2835 - sparse_categorical_accuracy: 0.9613
Epoch 6/10
938/938 [==============================] - 8s 9ms/step - loss: 0.2764 - sparse_categorical_accuracy: 0.9637
Epoch 7/10
938/938 [==============================] - 8s 9ms/step - loss: 0.2527 - sparse_categorical_accuracy: 0.9678
Epoch 8/10
938/938 [==============================] - 8s 9ms/step - loss: 0.2265 - sparse_categorical_accuracy: 0.9691
Epoch 9/10
938/938 [==============================] - 8s 9ms/step - loss: 0.2174 - sparse_categorical_accuracy: 0.9723
Epoch 10/10
938/938 [==============================] - 8s 9ms/step - loss: 0.2216 - sparse_categorical_accuracy: 0.9733

Out[6]:

<tensorflow.python.keras.callbacks.History at 0xf20a883340>

In [7]:

model.evaluate(test_dataset)

157/157 [==============================] - 1s 4ms/step - loss: 0.6802 - sparse_categorical_accuracy: 0.9594

Out[7]:

[0.6802026629447937, 0.9593999981880188]

In [ ]: