# Regression Analysis

## Linear Regression

`axle.ml.LinearRegression` makes use of `axle.algebra.LinearAlgebra`.

### Example: Home Prices

``````case class RealtyListing(size: Double, bedrooms: Int, floors: Int, age: Int, price: Double)

val listings = List(
RealtyListing(2104, 5, 1, 45, 460d),
RealtyListing(1416, 3, 2, 40, 232d),
RealtyListing(1534, 3, 2, 30, 315d),
RealtyListing(852, 2, 1, 36, 178d))``````

Create a price estimator using linear regression.

``````import cats.implicits._
import spire.algebra.Rng
import spire.algebra.NRoot
import axle.jblas._

implicit val rngDouble: Rng[Double] = spire.implicits.DoubleAlgebra
implicit val nrootDouble: NRoot[Double] = spire.implicits.DoubleAlgebra
implicit val rngInt: Rng[Int] = spire.implicits.IntAlgebra

import axle.ml.LinearRegression

val priceEstimator = LinearRegression(
listings,
numFeatures = 4,
featureExtractor = (rl: RealtyListing) => (rl.size :: rl.bedrooms.toDouble :: rl.floors.toDouble :: rl.age.toDouble :: Nil),
objectiveExtractor = (rl: RealtyListing) => rl.price,
α = 0.1,
iterations = 100)``````

Use the estimator

``````priceEstimator(RealtyListing(1416, 3, 2, 40, 0d))
// res0: Double = 288.60017635814035``````

Create a Plot of the error during the training

``````import axle.visualize._
import axle.algebra.Plottable._

val errorPlot = Plot(
() => List(("error" -> priceEstimator.errTree)),
connect = true,
drawKey = true,
colorOf = (label: String) => Color.black,
title = Some("Linear Regression Error"),
xAxis = Some(0d),
xAxisLabel = Some("step"),
yAxis = Some(0),
yAxisLabel = Some("error"))``````

Create the SVG

``````import axle.web._
import cats.effect._

errorPlot.svg[IO]("docwork/images/lrerror.svg").unsafeRunSync()``````

## Logistic Regression

WARNING: implementation is incorrect

`axle.ml.LogisticRegression` makes use of `axle.algebra.LinearAlgebra`.

### Example: Test Pass Probability

Predict Test Pass Probability as a Function of Hours Studied

``````case class Student(hoursStudied: Double, testPassed: Boolean)

val data = List(
Student(0.50, false),
Student(0.75, false),
Student(1.00, false),
Student(1.25, false),
Student(1.50, false),
Student(1.75, false),
Student(1.75, true),
Student(2.00, false),
Student(2.25, true),
Student(2.50, false),
Student(2.75, true),
Student(3.00, false),
Student(3.25, true),
Student(3.50, false),
Student(4.00, true),
Student(4.25, true),
Student(4.50, true),
Student(4.75, true),
Student(5.00, true),
Student(5.50, true)
)``````

Create a test pass probability function using logistic regression.

``````import spire.algebra.Rng
import spire.algebra.NRoot
import axle.jblas._

implicit val rngDouble: Rng[Double] = spire.implicits.DoubleAlgebra
// rngDouble: Rng[Double] = spire.std.DoubleAlgebra@3825d648
implicit val nrootDouble: NRoot[Double] = spire.implicits.DoubleAlgebra
// nrootDouble: NRoot[Double] = spire.std.DoubleAlgebra@3825d648
// laJblasDouble: axle.algebra.LinearAlgebra[org.jblas.DoubleMatrix, Int, Int, Double] = axle.jblas.package\$\$anon\$5@469c0c69
implicit val rngInt: Rng[Int] = spire.implicits.IntAlgebra
// rngInt: Rng[Int] = spire.std.IntAlgebra@5a0d8dc7

import axle.ml.LogisticRegression

val featureExtractor = (s: Student) => (s.hoursStudied :: Nil)
// featureExtractor: Student => List[Double] = <function1>

val objectiveExtractor = (s: Student) => s.testPassed
// objectiveExtractor: Student => Boolean = <function1>

val pTestPass = LogisticRegression(
data,
1,
featureExtractor,
objectiveExtractor,
0.1,
10)
// pTestPass: LogisticRegression[Student, org.jblas.DoubleMatrix] = LogisticRegression(
//   examples = List(
//     Student(hoursStudied = 0.5, testPassed = false),
//     Student(hoursStudied = 0.75, testPassed = false),
//     Student(hoursStudied = 1.0, testPassed = false),
//     Student(hoursStudied = 1.25, testPassed = false),
//     Student(hoursStudied = 1.5, testPassed = false),
//     Student(hoursStudied = 1.75, testPassed = false),
//     Student(hoursStudied = 1.75, testPassed = true),
//     Student(hoursStudied = 2.0, testPassed = false),
//     Student(hoursStudied = 2.25, testPassed = true),
//     Student(hoursStudied = 2.5, testPassed = false),
//     Student(hoursStudied = 2.75, testPassed = true),
//     Student(hoursStudied = 3.0, testPassed = false),
//     Student(hoursStudied = 3.25, testPassed = true),
//     Student(hoursStudied = 3.5, testPassed = false),
//     Student(hoursStudied = 4.0, testPassed = true),
//     Student(hoursStudied = 4.25, testPassed = true),
//     Student(hoursStudied = 4.5, testPassed = true),
//     Student(hoursStudied = 4.75, testPassed = true),
//     Student(hoursStudied = 5.0, testPassed = true),
//     Student(hoursStudied = 5.5, testPassed = true)
//   ),
//   numFeatures = 1,
//   featureExtractor = <function1>,
//   objectiveExtractor = <function1>,
//   α = 0.1,
//   numIterations = 10
// )``````

Use the estimator

``testPassProbability(2d :: Nil)``

(Note: The implementation is incorrect, so the result is elided until the error is fixed)

## Future Work

Fix Logistic Regression