雲端計算

Tensorflow Python 3.5 file

Tensorflow

Load the necessary libraries and import data from __future__ import print_function import math from IPython import display from matplotlib import cm from matplotlib import gridspec from matplotlib import pyplot as plt import numpy as np import pandas as pd from sklearn import metrics import tensorflow as tf from tensorflow.python.data import Dataset tf.logging.set_verbosity(tf.logging.ERROR) pd.options.display.max_rows = 10 pd.options.display.float_format = '{:.1f}'.format california_housing_dataframe = pd.read_csv("california_housing_train.csv", sep=",") california_housing_dataframe = california_housing_dataframe.reindex( np.random.permutation(california_housing_dataframe.index))

Prepares input features from data set def preprocess_features(california_housing_dataframe): selected_features = california_housing_dataframe[ [“latitude”, “longitude”, “housing_median_age”, "total_rooms", “total_bedrooms”, “population”, “households”, "median_income"]] processed_features = selected_features.copy() # Create a synthetic feature. processed_features["rooms_per_person"] = ( california_housing_dataframe["total_rooms"] / california_housing_dataframe["population"]) return processed_features 複製selected features 加入新的feature

Prepares target features (labels) from data set def preprocess_targets(california_housing_dataframe): output_targets = pd.DataFrame() # Scale the target to be in units of thousands of dollars. output_targets["median_house_value"] = ( california_housing_dataframe["median_house_value"] / 1000.0) return output_targets 建立空的 Daraframe 定義target feature

Training set Choose the first 12000 examples, out of the total of 17000. training_examples = preprocess_features(california_housing_dataframe.head(12000)) print(training_examples.describe()) 把前12000筆資料當作training set

Training targets (Label) Choose the first 12000 examples, out of the total of 17000. training_targets = preprocess_targets(california_housing_dataframe.head(12000)) print(training_targets.describe())

 Validation set Choose the last 5000 examples, out of the total of 17000. validation_examples = preprocess_features(california_housing_dataframe.tail(5000)) print(validation_examples.describe()) 把最後5000筆資料當作Validation set

 Validation target Choose the last 5000 examples, out of the total of 17000. validation_targets = preprocess_targets(california_housing_dataframe.tail(5000)) print(validation_targets.describe())

Construct the tensorflow feature columns def construct_feature_columns(input_features): """Construct the TensorFlow Feature Columns. Args: input_features: The names of the numerical input features to use. Returns: A set of feature columns """ return set([tf.feature_column.numeric_column(my_feature) for my_feature in input_features]) Name of numerical input features set():建立一個無序不重複元素的集合

Define input function def my_input_fn(features, targets, batch_size=1, shuffle=True, num_epochs=None): # Convert pandas data into a dict of np arrays. features = {key:np.array(value) for key,value in dict(features).items()} # Construct a dataset, and configure batching/repeating. ds = Dataset.from_tensor_slices((features,targets)) # warning: 2GB limit ds = ds.batch(batch_size).repeat(num_epochs) # Shuffle the data, if specified. if shuffle: ds = ds.shuffle(10000) # Return the next batch of data. features, labels = ds.make_one_shot_iterator().get_next() return features, labels

Train model with FTRL Optimization

Bucketized (Binned) features Bucketize population into 3 buckets bucket_0 (< 5000): corresponding to less populated blocks bucket_1 (5000 - 25000): corresponding to mid populated blocks bucket_2 (> 25000): corresponding to highly populated blocks Population vector: [[10001], [42004], [2500], [18000]] Bucketized feature vector: [[1], [2], [0], [1]]

Defines bucketized feature columns Bucketized_column takes a numeric column as input and transforms it to a bucketized feature Calculates boundaries based on quantiles

Train the model on bucketized feature columns

Train the model using feature crosses

Train the model using feature crosses

Python 3.5 file

Write file

Read file

Read file

驗收 建立test.txt，寫入資料 再將test.txt加入一行新的字串