1. 雲端計算

2. Tensorflow
Python 3.5 file

3. Tensorflow

5. Load the necessary libraries and import data
from __future__ import print_function
import math
from IPython import display
from matplotlib import cm
from matplotlib import gridspec
from matplotlib import pyplot as plt
import numpy as np
import pandas as pd
from sklearn import metrics
import tensorflow as tf
from tensorflow.python.data import Dataset
tf.logging.set_verbosity(tf.logging.ERROR)
pd.options.display.max_rows = 10
pd.options.display.float_format = '{:.1f}'.format
california_housing_dataframe = pd.read_csv("california_housing_train.csv", sep=",")
california_housing_dataframe = california_housing_dataframe.reindex(
np.random.permutation(california_housing_dataframe.index))

6. Prepares input features from data set
def preprocess_features(california_housing_dataframe):
selected_features = california_housing_dataframe[
[“latitude”, “longitude”, “housing_median_age”, "total_rooms",
“total_bedrooms”, “population”, “households”, "median_income"]]
processed_features = selected_features.copy()
# Create a synthetic feature.
processed_features["rooms_per_person"] = (
california_housing_dataframe["total_rooms"] /
california_housing_dataframe["population"])
return processed_features
複製selected features
加入新的feature

7. Prepares target features (labels) from data set
def preprocess_targets(california_housing_dataframe):
output_targets = pd.DataFrame()
# Scale the target to be in units of thousands of dollars.
output_targets["median_house_value"] = (
california_housing_dataframe["median_house_value"] / )
return output_targets
建立空的 Daraframe
定義target feature

8. Training set
Choose the first examples, out of the total of
training_examples = preprocess_features(california_housing_dataframe.head(12000))
print(training_examples.describe())
把前12000筆資料當作training set

9. Training targets (Label)
Choose the first examples, out of the total of
training_targets = preprocess_targets(california_housing_dataframe.head(12000))
print(training_targets.describe())

10.  Validation set
Choose the last 5000 examples, out of the total of
validation_examples = preprocess_features(california_housing_dataframe.tail(5000))
print(validation_examples.describe())
把最後5000筆資料當作Validation set

11.  Validation target
Choose the last 5000 examples, out of the total of
validation_targets = preprocess_targets(california_housing_dataframe.tail(5000))
print(validation_targets.describe())

12. Construct the tensorflow feature columns
def construct_feature_columns(input_features):
"""Construct the TensorFlow Feature Columns.
Args:
input_features: The names of the numerical input features to use.
Returns:
A set of feature columns
"""
return set([tf.feature_column.numeric_column(my_feature)
for my_feature in input_features])
Name of numerical input features
set():建立一個無序不重複元素的集合

13. Define input function
def my_input_fn(features, targets, batch_size=1, shuffle=True, num_epochs=None):
# Convert pandas data into a dict of np arrays.
features = {key:np.array(value) for key,value in dict(features).items()}
# Construct a dataset, and configure batching/repeating.
ds = Dataset.from_tensor_slices((features,targets)) # warning: 2GB limit
ds = ds.batch(batch_size).repeat(num_epochs)
# Shuffle the data, if specified.
if shuffle:
ds = ds.shuffle(10000)
# Return the next batch of data.
features, labels = ds.make_one_shot_iterator().get_next()
return features, labels

14. Train model with FTRL Optimization

15. Bucketized (Binned) features
Bucketize population into 3 buckets
bucket_0 (< 5000): corresponding to less populated blocks
bucket_1 ( ): corresponding to mid populated blocks
bucket_2 (> 25000): corresponding to highly populated blocks
Population vector:
[[10001], [42004], [2500], [18000]]
Bucketized feature vector:
[[1], [2], [0], [1]]

16. Defines bucketized feature columns
Bucketized_column takes a numeric column as input and transforms it to a bucketized feature
Calculates boundaries based on quantiles

17. Train the model on bucketized feature columns

18. Train the model using feature crosses

19. Train the model using feature crosses

20. Python 3.5 file

21. Write file

22. Read file

23. Read file

24. 驗收
建立test.txt，寫入資料
再將test.txt加入一行新的字串