1. Week 1 - An Introduction to Machine Learning & Soft Computing
-Yosi Kristian-

2. Soft Computing STTS – Yosi Kristian

3. Definition
Soft Computing is a term applied to a field within computer science which is characterized by the use of inexact solutions to computationally hard tasks, for which there is no known algorithm that can compute an exact solution. 
Soft computing differs from conventional (hard) computing in that, unlike hard computing, it is tolerant of imprecision, uncertainty, partial truth, and approximation.
Soft Computing STTS – Yosi Kristian

4. Still the Definition..
In effect, the role model for soft computing is the human mind.
The guiding principle of soft computing is: Exploit the tolerance for imprecision, uncertainty, partial truth, and approximation to achieve tractability, robustness and low solution cost.
Soft Computing STTS – Yosi Kristian

5. Components of soft computing
What ???
Are we going to learn them all
in this subject?
Neural networks (NN)
Support Vector Machines (SVM)
Fuzzy logics (FL)
Evolutionary computation (EC), including:
Evolutionary algorithms
Genetic algorithms
Differential evolution
Meta heuristic and Swarm Intelligence
Ant colony optimization
Bees algorithms
Bat algorithm
Cuckoo search
Harmony search
Firefly algorithm
Artificial immune systems
Particle swarm optimization
Soft Computing STTS – Yosi Kristian

6. Soft Computing in AI
Soft computing may be viewed as a foundation component for the emerging field of conceptual intelligence.
Machine Learning
Fuzzy Systems
Evolutionary Computation
Probabilistic Reasoning
Soft Computing is the CORE component of many Machine Learning System
Soft Computing STTS – Yosi Kristian

7. Machine Learning
Arthur Samuel (1959). Machine Learning: Field of study that gives computers the ability to learn without being explicitly programmed.
Tom Mitchell (1998) Well-posed Learning Problem: A computer program is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E.
Soft Computing STTS – Yosi Kristian

8. Machine learning usage
Usage of Machine Learning is to develop applications that can’t be programed by hand.
E.g., Autonomous helicopter, handwriting recognition, most of Natural Language Processing (NLP), Computer Vision etc.
Or a machine learning system could be trained on messages to learn to distinguish between spam and non-spam messages. After learning, it can then be used to classify new messages into spam and non-spam folders.
Soft Computing STTS – Yosi Kristian

9. Machine Learning Categorized By Data and Learning Process
Soft Computing STTS – Yosi Kristian

10. Soft Computing In Machine Learning
Soft Computing is the soul of many machine learning system.
Classification and Clustering is a very common soft computing problems.
Soft Computing STTS – Yosi Kristian

11. Intro to Supervised Learning
Soft Computing STTS – Yosi Kristian

12. Example Housing price prediction. Supervised Learning
in 1000’s
Size in feet2
Supervised Learning
“right answers” given
Regression: Predict continuous valued output (price)
Soft Computing STTS – Yosi Kristian

13. Example Breast cancer (malignant, benign) Classification
Discrete valued output (0 or 1)
1(Y)
Malignant?
0(N)
Tumor Size
Tumor Size
Soft Computing STTS – Yosi Kristian

14. Another Example Clump Thickness Uniformity of Cell Size
Uniformity of Cell Shape …
Age
Tumor Size
Soft Computing STTS – Yosi Kristian

15. Exercise
You’re running a company, and you want to develop learning algorithms to address each of two problems.
Problem 1: You have a large inventory of identical items. You want to predict how many of these items will sell over the next 3 months.
Problem 2: You’d like software to examine individual customer accounts, and for each account decide if it has been hacked/compromised.
Should you treat these as classification or as regression problems?
Soft Computing STTS – Yosi Kristian

16. Answer Treat both as classification problems.
Treat problem 1 as a classification problem, problem 2 as a regression problem.
Treat problem 1 as a regression problem, problem 2 as a classification problem.
Treat both as regression problems.
Soft Computing STTS – Yosi Kristian

17. Classification Example
Another Example is for image Classification / Categorization
Training
Training Labels
Training Images
Image Features
Classifier Training
Trained Classifier
Soft Computing STTS – Yosi Kristian

18. Cont… Testing Image Features Trained Classifier Test Image Prediction
Outdoor
Soft Computing STTS – Yosi Kristian

19. Learning a classifier
Given some set of features with corresponding labels, learn a function to predict the labels from the features
Training labels dictate that two examples are the same or different, in some sense
Features and distance measures define similarity
Classifiers try to learn weights or parameters for features and distance measures so that feature similarity predicts label similarity
Soft Computing STTS – Yosi Kristian

20. Intro to Unsupervised Learning
Soft Computing STTS – Yosi Kristian

21. Supervised Learning x2 x1
Soft Computing STTS – Yosi Kristian

22. Unsupervised Learningx2 x1
Soft Computing STTS – Yosi Kristian

23. Clustering Example
Soft Computing STTS – Yosi Kristian

24. Contd…
Soft Computing STTS – Yosi Kristian

25. Exercise
Of the following examples, which would you address using an unsupervised learning algorithm? (Check all that apply.)
Given labeled as spam/not spam, learn a spam filter.
Given a set of news articles found on the web, group them into set of articles about the same story.
Given a database of customer data, automatically discover market segments and group customers into different market segments.
Given a dataset of patients diagnosed as either having diabetes or not, learn to classify new patients as having diabetes or not.
Soft Computing STTS – Yosi Kristian

26. Warming Up….
Do 10 x Push Ups.
Soft Computing STTS – Yosi Kristian

27. Linear Regression with one variable
Housing Prices
(Portland, OR)
Price
(in 1000s of dollars)
Size (feet2)
Supervised Learning
Given the “right answer” for each example in the data.
Regression Problem
Predict real-valued output
Soft Computing STTS – Yosi Kristian

28. Linear Regression with one variable
Training set of
housing prices
(Portland, OR)
Notation:
m = Number of training examples
n = Number of feature
x’s = “input” variable / features
y’s = “output” variable / “target” variable
Soft Computing STTS – Yosi Kristian

29. The Concept Training Set How do we represent h ? Learning Algorithm
Size of house h
Estimated price
Linear regression with one variable.
Univariate linear regression.
Soft Computing STTS – Yosi Kristian

30. Cost Function Training Set Hypothesis: ‘s: Parameters
How to choose ‘s ?
Soft Computing STTS – Yosi Kristian

31. Contd..
Soft Computing STTS – Yosi Kristian

32. Cost Function.. y x
Idea: Choose so that is close to for our training examples
Soft Computing STTS – Yosi Kristian

33. Simplification: For the sake of understanding
Simplified
Hypothesis:
Parameters:
Cost Function:
Goal:
Soft Computing STTS – Yosi Kristian

34. Trial 1 y x (function of the parameter )
(for fixed , this is a function of x) y x
Soft Computing STTS – Yosi Kristian

35. Trial 2 y x (for fixed , this is a function of x)
(function of the parameter ) y x
Soft Computing STTS – Yosi Kristian

36. Trial 3 y x (function of the parameter )
(for fixed , this is a function of x) y x
Soft Computing STTS – Yosi Kristian

37. Done with simplification, back to real world.
Hypothesis:
Parameters:
Cost Function:
Goal:
Soft Computing STTS – Yosi Kristian

38. The Contour Figures …. How to find minimum of J in that?
Soft Computing STTS – Yosi Kristian

39. Gradient Descent..
Next Week…
Soft Computing STTS – Yosi Kristian