{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "header",
   "metadata": {},
   "source": [
    "# Solving Systems of Linear Equations\n",
    "\n",
    "Matrices provide a compact way to solve multiple equations at once. Instead of solving equations one by one, we can solve them all together using matrix operations.\n",
    "\n",
    "## The Problem\n",
    "\n",
    "Suppose you have a system of linear equations:\n",
    "\n",
    "$$\n",
    "2x + 3y &= 8 \\\\\n",
    "x - y &= 1\n",
    "$$\n",
    "\n",
    "We can write this as a matrix equation:\n",
    "\n",
    "$$\n",
    "A\\mathbf{x} = \\mathbf{b}\n",
    "$$\n",
    "\n",
    "Where:\n",
    "\n",
    "$$\n",
    "A = \\begin{bmatrix}\n",
    "2 & 3 \\\\\n",
    "1 & -1\n",
    "\\end{bmatrix}, \\quad\n",
    "\\mathbf{x} = \\begin{bmatrix}\n",
    "x \\\\\n",
    "y\n",
    "\\end{bmatrix}, \\quad\n",
    "\\mathbf{b} = \\begin{bmatrix}\n",
    "8 \\\\\n",
    "1\n",
    "\\end{bmatrix}\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "setup",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "solution",
   "metadata": {},
   "source": [
    "## Solving with NumPy\n",
    "\n",
    "NumPy provides `np.linalg.solve()` to solve $A\\mathbf{x} = \\mathbf{b}$:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "solve-example",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define the system\n",
    "A = np.array([[2, 3],\n",
    "              [1, -1]])\n",
    "\n",
    "b = np.array([8, 1])\n",
    "\n",
    "# Solve for x\n",
    "x = np.linalg.solve(A, b)\n",
    "\n",
    "print(f\"Solution: x = {x[0]:.2f}, y = {x[1]:.2f}\")\n",
    "\n",
    "# Verify the solution\n",
    "print(f\"\\nVerification:\")\n",
    "print(f\"2({x[0]:.2f}) + 3({x[1]:.2f}) = {2*x[0] + 3*x[1]:.2f} (should be 8)\")\n",
    "print(f\"({x[0]:.2f}) - ({x[1]:.2f}) = {x[0] - x[1]:.2f} (should be 1)\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "visualization",
   "metadata": {},
   "source": [
    "## Geometric Interpretation\n",
    "\n",
    "Each equation represents a line. The solution is where the lines intersect:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "plot-lines",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Plot the two equations as lines\n",
    "x_vals = np.linspace(-2, 6, 100)\n",
    "\n",
    "# 2x + 3y = 8  =>  y = (8 - 2x) / 3\n",
    "y1 = (8 - 2*x_vals) / 3\n",
    "\n",
    "# x - y = 1  =>  y = x - 1\n",
    "y2 = x_vals - 1\n",
    "\n",
    "plt.figure(figsize=(8, 6))\n",
    "plt.plot(x_vals, y1, label='2x + 3y = 8', linewidth=2)\n",
    "plt.plot(x_vals, y2, label='x - y = 1', linewidth=2)\n",
    "plt.scatter(x[0], x[1], color='red', s=100, zorder=5, \n",
    "            label=f'Solution: ({x[0]:.2f}, {x[1]:.2f})')\n",
    "plt.xlabel('x')\n",
    "plt.ylabel('y')\n",
    "plt.grid(True, alpha=0.3)\n",
    "plt.axhline(0, color='black', linewidth=0.5)\n",
    "plt.axvline(0, color='black', linewidth=0.5)\n",
    "plt.legend()\n",
    "plt.title('System of Equations: Intersection is the Solution')\n",
    "plt.xlim(-1, 5)\n",
    "plt.ylim(-1, 4)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "larger-system",
   "metadata": {},
   "source": [
    "## Larger Systems\n",
    "\n",
    "The same approach works for larger systems. Let's solve a 3x3 system:\n",
    "\n",
    "$$\n",
    "x + 2y + z &= 6 \\\\\n",
    "2x + y - z &= 3 \\\\\n",
    "x - y + 2z &= 5\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "solve-3x3",
   "metadata": {},
   "outputs": [],
   "source": [
    "A = np.array([[1, 2, 1],\n",
    "              [2, 1, -1],\n",
    "              [1, -1, 2]])\n",
    "\n",
    "b = np.array([6, 3, 5])\n",
    "\n",
    "x = np.linalg.solve(A, b)\n",
    "\n",
    "print(f\"Solution: x = {x[0]:.2f}, y = {x[1]:.2f}, z = {x[2]:.2f}\")\n",
    "\n",
    "# Verify\n",
    "print(f\"\\nVerification: A @ x = {A @ x}\")\n",
    "print(f\"Should equal b = {b}\")\n",
    "print(f\"Difference: {np.linalg.norm(A @ x - b):.2e}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "inverse",
   "metadata": {},
   "source": [
    "## Using the Inverse Matrix\n",
    "\n",
    "Mathematically, the solution is:\n",
    "\n",
    "$$\n",
    "\\mathbf{x} = A^{-1}\\mathbf{b}\n",
    "$$\n",
    "\n",
    "Where $A^{-1}$ is the inverse of $A$. However, **don't compute the inverse explicitly** in practice - `np.linalg.solve()` is faster and more numerically stable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "inverse-demo",
   "metadata": {},
   "outputs": [],
   "source": [
    "A = np.array([[2, 3],\n",
    "              [1, -1]])\n",
    "b = np.array([8, 1])\n",
    "\n",
    "# Using inverse (not recommended)\n",
    "A_inv = np.linalg.inv(A)\n",
    "x_inverse = A_inv @ b\n",
    "\n",
    "# Using solve (recommended)\n",
    "x_solve = np.linalg.solve(A, b)\n",
    "\n",
    "print(\"Using inverse:\")\n",
    "print(f\"x = {x_inverse}\")\n",
    "print(\"\\nUsing solve:\")\n",
    "print(f\"x = {x_solve}\")\n",
    "print(\"\\nBoth give the same answer, but solve() is better!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "no-solution",
   "metadata": {},
   "source": [
    "## When There's No Solution\n",
    "\n",
    "If the system has no solution (parallel lines that don't intersect), the matrix is **singular** (not invertible):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "no-solution-demo",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Parallel lines: no intersection\n",
    "A = np.array([[2, 4],\n",
    "              [1, 2]])  # Second row is half of first row\n",
    "\n",
    "b = np.array([8, 5])  # Inconsistent right-hand side\n",
    "\n",
    "try:\n",
    "    x = np.linalg.solve(A, b)\n",
    "    print(f\"Solution: {x}\")\n",
    "except np.linalg.LinAlgError as e:\n",
    "    print(f\"Error: {e}\")\n",
    "    print(\"\\nThe system has no solution - the lines are parallel!\")\n",
    "\n",
    "# Check if matrix is singular\n",
    "det = np.linalg.det(A)\n",
    "print(f\"\\nDeterminant of A: {det:.2e}\")\n",
    "print(\"Determinant is zero => matrix is singular\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "overdetermined",
   "metadata": {},
   "source": [
    "## Overdetermined Systems\n",
    "\n",
    "Sometimes you have more equations than unknowns (e.g., 3 equations, 2 unknowns). This is an **overdetermined system** and typically has no exact solution.\n",
    "\n",
    "For these cases, use {doc}`least-squares` to find the best approximate solution."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "real-world",
   "metadata": {},
   "source": [
    "## Real-World Example: Circuit Analysis\n",
    "\n",
    "Solving for currents in an electrical circuit using Kirchhoff's laws:\n",
    "\n",
    "$$\n",
    "I_1 + I_2 &= I_3 \\quad \\text{(current in = current out)} \\\\\n",
    "10I_1 + 5I_2 &= 15 \\quad \\text{(voltage loop 1)} \\\\\n",
    "5I_2 + 20I_3 &= 25 \\quad \\text{(voltage loop 2)}\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "circuit-example",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Circuit equations in matrix form\n",
    "A = np.array([[1, 1, -1],\n",
    "              [10, 5, 0],\n",
    "              [0, 5, 20]])\n",
    "\n",
    "b = np.array([0, 15, 25])\n",
    "\n",
    "I = np.linalg.solve(A, b)\n",
    "\n",
    "print(\"Circuit Currents:\")\n",
    "print(f\"I1 = {I[0]:.3f} A\")\n",
    "print(f\"I2 = {I[1]:.3f} A\")\n",
    "print(f\"I3 = {I[2]:.3f} A\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "summary",
   "metadata": {},
   "source": [
    "## Summary\n",
    "\n",
    "- Systems of linear equations can be written as $A\\mathbf{x} = \\mathbf{b}$\n",
    "- Use `np.linalg.solve(A, b)` to find the solution\n",
    "- The solution is where all equations (lines/planes) intersect\n",
    "- If no solution exists, the matrix is singular (determinant = 0)\n",
    "- For overdetermined systems, use least squares instead\n",
    "\n",
    "## Next Steps\n",
    "\n",
    "When exact solutions don't exist, we can still find the **best approximate solution** using {doc}`least-squares`."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.12.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}