neural_network/numpy_nn/partial_derivative_9.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "a8a2a463",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sympy import symbols, diff"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d52830dc",
   "metadata": {},
   "source": [
    "## 1.偏导(Partial Derivative)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e6d6f8f4",
   "metadata": {},
   "source": [
    "$$f(x,y,z)\\rightarrow \\frac{\\partial}{\\partial x}f(x,y,z), \\frac{\\partial}{\\partial y}f(x,y,z), \\frac{\\partial}{\\partial z}f(x,y,z)$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c17efaa6",
   "metadata": {},
   "source": [
    "偏导的计算和对单一的变量求导方式一致，对一个变量求偏导，其他变量看成常数。上一章的求导符号是$d$，在偏导中将使用$\\partial$符号。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eab1aa5d",
   "metadata": {},
   "source": [
    "## 2.偏导加和"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d78b4dd1",
   "metadata": {},
   "source": [
    "$$f(x, y) = x + y$$  \n",
    "$$\\frac{\\partial}{\\partial x}f(x, y)=\\frac{\\partial}{\\partial x} [x+y] = \\frac{\\partial}{\\partial x}x+\\frac{\\partial}{\\partial x}y = 1+0=1$$  \n",
    "$$\\frac{\\partial}{\\partial y}f(x, y)=\\frac{\\partial}{\\partial y} [x+y] = \\frac{\\partial}{\\partial y}x+\\frac{\\partial}{\\partial y}y=0+1=1$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "c1b55362",
   "metadata": {},
   "outputs": [],
   "source": [
    "x = symbols('x')\n",
    "y = symbols('y')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "85d0d127",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1, 1)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "diff(x+y, x), diff(x+y, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cdbf2e9a",
   "metadata": {},
   "source": [
    "$$f(x, y)=2x+3y^2$$  \n",
    "$$\\frac{\\partial}{\\partial x}f(x, y)=\\frac{\\partial}{\\partial x}[2x+3y^2] = \\frac{\\partial}{\\partial x}2x + \\frac{\\partial}{\\partial x}3y^2=2\\cdot\\frac{\\partial}{\\partial x}x + 3\\cdot\\frac{\\partial}{\\partial x}y^2=2\\cdot 1 + 3\\cdot 0 = 2$$  \n",
    "$$\\frac{\\partial}{\\partial y}f(x, y)=\\frac{\\partial}{\\partial y}[2x+3y^2] = \\frac{\\partial}{\\partial y}2x + \\frac{\\partial}{\\partial y}3y^2=2\\cdot\\frac{\\partial}{\\partial y}x + 3\\cdot\\frac{\\partial}{\\partial y}y^2=2\\cdot 0 + 3\\cdot 2y^1 = 6y$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "b32ffead",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2, 6*y)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "diff(2*x+3*y**2, x), diff(2*x+3*y**2, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dfd0d769",
   "metadata": {},
   "source": [
    "$$f(x, y)=3x^3-y^2+5x+2$$  \n",
    "$$\\frac{\\partial}{\\partial x}f(x, y)=\\frac{\\partial}{\\partial x}[3x^2-y^2+5x+2]=\\frac{\\partial}{\\partial x}3x^2-\\frac{\\partial}{\\partial x}y^2+\\frac{\\partial}{\\partial x}5x + \\frac{\\partial}{\\partial x}2=\\\\3\\cdot\\frac{\\partial}{\\partial x}x^3-\\frac{\\partial}{\\partial x}y^2+5\\cdot\\frac{\\partial}{\\partial x}x+\\frac{\\partial}{\\partial x}2=3\\cdot3x^2-0+5\\cdot 1 + 0 = 9x^2+5$$  \n",
    "$$\\frac{\\partial}{\\partial y}f(x, y)=\\frac{\\partial}{\\partial y}[3x^2-y^2+5x+2]=\\frac{\\partial}{\\partial y}3x^2-\\frac{\\partial}{\\partial y}y^2+\\frac{\\partial}{\\partial y}5x + \\frac{\\partial}{\\partial y}2=\\\\3\\cdot\\frac{\\partial}{\\partial y}x^3-\\frac{\\partial}{\\partial y}y^2+5\\cdot\\frac{\\partial}{\\partial y}x+\\frac{\\partial}{\\partial y}2=3\\cdot0-2y^1 + 5\\cdot 1+0 = -2y$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "2a5b70b4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(9*x**2 + 5, -2*y)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "diff(3*x**3-y**2+5*x+2, x), diff(3*x**3-y**2+5*x+2, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fa03d274",
   "metadata": {},
   "source": [
    "## 3.偏导乘"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "04569578",
   "metadata": {},
   "source": [
    "$$f(x, y)=x\\cdot y$$  \n",
    "$$\\frac{\\partial}{\\partial x}f(x, y)=\\frac{\\partial}{\\partial x}[x\\cdot y]=y\\frac{\\partial}{\\partial x}x = y\\cdot 1 = y$$  \n",
    "$$\\frac{\\partial}{\\partial y}f(x, y)=\\frac{\\partial}{\\partial y}[x\\cdot y]=x\\frac{\\partial}{\\partial y}y = x\\cdot 1 = x$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "01093efb",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(y, x)"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "diff(x*y, x),diff(x*y, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eda4cf95",
   "metadata": {},
   "source": [
    "$$f(x,y,z)=3x^2z-y^2+5z+2yz$$  \n",
    "$$\\frac{\\partial}{\\partial x}f(x,y,z)=\\frac{\\partial}{\\partial x}[3x^3z-y^2+5z+2yz]=\\\\\n",
    "\\frac{\\partial}{\\partial x}3x^3z-\\frac{\\partial}{\\partial x}y^2+\\frac{\\partial}{\\partial x}y^2 +\\frac{\\partial}{\\partial x}5z + \\frac{\\partial}{\\partial x}2yz=\\\\\n",
    "3z\\cdot\\frac{\\partial}{\\partial x}x^3-\\frac{\\partial}{\\partial x}y^2+5\\cdot\\frac{\\partial}{\\partial x}z+2\\cdot\\frac{\\partial}{\\partial x}yz=\\\\\n",
    "3z\\cdot3x^2-0+5\\cdot 0 + 2\\cdot 0=9x^2z$$  \n",
    "$$\\frac{\\partial}{\\partial y}f(x,y,z)=\\frac{\\partial}{\\partial y}[3x^3z-y^2+5z+2yz]=\\\\\n",
    "\\frac{\\partial}{\\partial y}3x^3z-\\frac{\\partial}{\\partial y}y^2+\\frac{\\partial}{\\partial y}y^2 +\\frac{\\partial}{\\partial y}5z + \\frac{\\partial}{\\partial y}2yz=\\\\\n",
    "3\\cdot\\frac{\\partial}{\\partial y}x^3z-\\frac{\\partial}{\\partial y}y^2+5\\cdot\\frac{\\partial}{\\partial y}z+2z\\cdot\\frac{\\partial}{\\partial y}y=\\\\\n",
    "3\\cdot 0- 2y + 5\\cdot 0 + 2z\\cdot 1=-2y + 2z$$  \n",
    "$$\\frac{\\partial}{\\partial z}f(x,y,z)=\\frac{\\partial}{\\partial z}[3x^3z-y^2+5z+2yz]=\\\\\n",
    "\\frac{\\partial}{\\partial z}3x^3z-\\frac{\\partial}{\\partial z}y^2+\\frac{\\partial}{\\partial z}y^2 +\\frac{\\partial}{\\partial z}5z + \\frac{\\partial}{\\partial z}2yz=\\\\\n",
    "3x^3\\cdot\\frac{\\partial}{\\partial z}z-\\frac{\\partial}{\\partial z}y^2+5\\cdot\\frac{\\partial}{\\partial z}z+2y\\cdot\\frac{\\partial}{\\partial z}z=\\\\\n",
    "3x^3\\cdot 1- 0 + 5\\cdot 1 + 2y\\cdot 1=3x^3+5+2y$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "35595819",
   "metadata": {},
   "outputs": [],
   "source": [
    "z = symbols('z')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "15db90db",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(9*x**2*z, -2*y + 2*z, 3*x**3 + 2*y + 5)"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "diff(3*x**3*z-y**2+5*z+2*y*z, x),diff(3*x**3*z-y**2+5*z+2*y*z, y), diff(3*x**3*z-y**2+5*z+2*y*z, z)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "258507b5",
   "metadata": {},
   "source": [
    "## 4.对Max函数求偏导"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1debf836",
   "metadata": {},
   "source": [
    "$$f(x,y)=max(x,y)$$  \n",
    "$$\\frac{\\partial}{\\partial x}f(x,y)=\\frac{\\partial}{\\partial x}max(x,y)=1(x > y)$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "d2c73de9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/latex": [
       "$\\displaystyle 1$"
      ],
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "diff(x, x)#x > y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "e28acd91",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/latex": [
       "$\\displaystyle 0$"
      ],
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "diff(y, x)#x <=y"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dcb06400",
   "metadata": {},
   "source": [
    "ReLU激活函求导，因为ReLU函数只有一个变量x，所以这里叫求导，不叫求偏导。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b5c6ab03",
   "metadata": {},
   "source": [
    "$$f(x)=max(x,0)$$  \n",
    "$$\\frac{d}{d x}f(x,y)=\\frac{d}{d x}max(x,0)=1(x > 0)$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "2aa7a2be",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/latex": [
       "$\\displaystyle 1$"
      ],
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "diff(x, x)#x > 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "6977b706",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/latex": [
       "$\\displaystyle 0$"
      ],
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "diff(0, x)#x < 0"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}