# Number Theory: Divisibility

This is the first note in the series Number Theory: Divisibility. All numbers involved in this note are integers, and letters used in this note stand for integers without further specification.

Numbers involved in this note are integers, and letters used in this book stand for integers without further specification.

Given numbers $$a$$ and $$b$$, with $$b \neq 0$$, if there is an integer $$c$$, such that $$a=bc$$, then we say $$b$$ divides $$a$$, and write $$b \mid a$$. In this case we also say $$b$$ is a factor of $$a$$, or $$a$$ is a multiple of $$b$$. We use the notation $$b \nmid a$$ when $$b$$ does not divide $$a$$ (i.e., no such $$c$$ exists).

Several simple properties of divisibility could be obtained by the definition of divisibility (proofs of the properties are left to readers).

$$(1)$$ If $$b \mid c,$$ and $$c \mid a,$$ then $$b \mid a,$$ that is, divisibility is transitive.

$$(2)$$ If $$b \mid a,$$ and $$b \mid c,$$ then $$b \mid (a \pm c),$$ that is, the set of multiples of an integer is closed under addition and subtraction operations.

By using this property repeatedly, we have, if $$b \mid a$$ and $$b \mid c$$, then $$b \mid (au+cv)$$, for any integers $$u$$ and $$v$$. In general, if $$a_1,a_2,\cdots,a_n$$ are multiples of $$b$$, then $b \mid (a_1+a_2+\cdots+a_n).$

$$(3)$$ If $$b \mid a$$, then $$a=0$$ or $$\lvert a \rvert \geq \lvert b \rvert$$. Thus, if $$b \mid a$$ and $$a \mid b$$, then $$\lvert a \rvert = \lvert b \rvert$$.

Clearly, for any two integers $$a$$ and $$b$$, $$a$$ is not always divisible by $$b$$. But we have the following result, which is called the division algorithm. It is the most important result in elementary number theory.

$$(4)$$ (The division algorithm) Let $$a$$ and $$b$$ be integers, and $$b > 0$$. Then there is a unique pair of integers $$q$$ and $$r$$, such that $a=bq+r \hspace{2mm} \text{and} \hspace{2mm} 0 \leq r < b.$

The integer $$q$$ is called the (incomplete) quotient when $$a$$ is divided by $$b$$, $$r$$ called the remainder. Note that the values of $$r$$ has $$b$$ kinds of possibilities, $$0,1,\cdots,b-1$$. If $$r=0$$, then $$a$$ is divisible by $$b$$.

It is easy to see that the quotient $$q$$ in the division algorithm is in fact $$\lfloor\frac{a}{b}\rfloor$$ (the greatest integer not exceeding $$\frac{a}{b}$$), and the heart of the division algorithm is the inequality about the remainder $$r$$: $$0 \leq r < b$$. We will go back to this point later on.

The basic method of proving $$b \mid a$$ is to factorize $$a$$ into the product of $$b$$ and another integer. Usually, in some basic problems this kind of factorization can be obtained by taking some special value in algebraic factorization equations. The following two factorization formulae are very useful in proving this kind of problems.

$$(5)$$ If $$n$$ is a positive integer, then $x^n-y^n=(x-y)(x^{n-1}+x^{n-2}y+\cdots+xy^{n-2}+y^{n-1}).$

$$(6)$$ If $$n$$ is a positive odd number, then $x^n+y^n=(x+y)(x^{n-1}-x^{n-2}y+\cdots-xy^{n-2}+y^{n-1}).$

Note by Victor Loh
3 years, 8 months ago

MarkdownAppears as
*italics* or _italics_ italics
**bold** or __bold__ bold

- bulleted
- list

• bulleted
• list

1. numbered
2. list

1. numbered
2. list
Note: you must add a full line of space before and after lists for them to show up correctly
paragraph 1

paragraph 2

paragraph 1

paragraph 2

> This is a quote
This is a quote
# I indented these lines
# 4 spaces, and now they show
# up as a code block.

print "hello world"
# I indented these lines
# 4 spaces, and now they show
# up as a code block.

print "hello world"
MathAppears as
Remember to wrap math in $$...$$ or $...$ to ensure proper formatting.
2 \times 3 $$2 \times 3$$
2^{34} $$2^{34}$$
a_{i-1} $$a_{i-1}$$
\frac{2}{3} $$\frac{2}{3}$$
\sqrt{2} $$\sqrt{2}$$
\sum_{i=1}^3 $$\sum_{i=1}^3$$
\sin \theta $$\sin \theta$$
\boxed{123} $$\boxed{123}$$

Sort by:

Thanks much for posting this. I would love if you could name the book you had referred to in the beginning of this note!

- 3 years, 8 months ago

I'm not sure if I want to reveal it because I'm scared I'm taking a little too much information :D

- 3 years, 8 months ago

I like the fact that although it's meant to be a comment with a negative connotation, there's still a smiley icon at the back which doesn't really fit the sentence (lol). Also, I think I have an idea which book he may have referred to...

- 3 years, 8 months ago

But still, we have to acknowledge the rules of copyright. IF I feel like checking this book and find that you have really gleaned too much information from it, I might have to report you... D:

- 3 years, 8 months ago

It's not THAT book, I promise.

- 3 years, 8 months ago

Anyway, Brilliant is a site for sharing information, and I haven't exactly shared the examples and exercises yet.

- 3 years, 8 months ago

Uh-Oh!

- 3 years, 8 months ago


# Confusion with Matrix calculus derivative computation.

The text I am reading has the following

$$E=(I+\nabla I\frac{\partial W}{\partial p}\Delta p -T)^2$$ where $\nabla I$ us a row vector $1$ by $2$, $\frac{\partial W}{\partial p}$ is a $2$ by $3$ matrix and $\Delta p$ is a $3$ by $1$ column vector. $I$ and $T$ are scalars. We seek the gradient $\frac{\partial E}{\partial \Delta p_k}$, but I do not understand their solution. I will present my attempt, and than their solution.

In index notation, we have $(I+\nabla I_i\frac{\partial W^i}{\partial \Delta p_k}\Delta p^k -T)^2$, where summation if implied by repeated indicies, Than the derivative is

$$\frac{\partial E}{\partial \Delta p_m}=2(I+\nabla I_i\frac{\partial W^i}{\partial p_k}\Delta p^k -T)\nabla I_n\frac{\partial W^n}{\partial p_m}$$ or returning to the matrix notation

$$\frac{\partial E}{\partial \Delta p}=2(I+\nabla I\frac{\partial W}{\partial p}\Delta p -T)\nabla I\frac{\partial W}{\partial p}\tag{1}\label{1}$$

But their solution is

$$\frac{\partial E}{\partial \Delta p}=2(\nabla I\frac{\partial W}{\partial p})^T(I+\nabla I\frac{\partial W}{\partial p}\Delta p -T)\tag{2}\label{2}$$

Why is their solution correct and where did I make the mistake? My solution is equation \eqref{1}, but theirs is \eqref{2}. I am confused, why in theirs, $$(\nabla I\frac{\partial W}{\partial p})$$ is transposed?

• In the last two equations you have variable $p_m$ with subindex $m$ on the left hand side, but no subscripts on the right hand side. Is there a chance there is a typo in your problem statement?
Mar 21 '18 at 2:30
• I was careless when copy and pasting. My solution in matrix form is that in equation (1), what is before, is my derivation. Their solution is equation (2). Hope this clears things up Mar 21 '18 at 2:33
• I think now you very first equation (the one without label) has non-matching subindex $p_m$ in the last term.
Mar 21 '18 at 2:35
• You are correct, I have fixed it. Thank you for sticking with me, it has been a long day :) Mar 21 '18 at 2:36
• I edited and added that information. Mar 21 '18 at 2:52

It looks like the text you are reading uses so-called Denominator layout, for matrix calculus notation, i.e. given two column vectors $\boldsymbol{x}\in\mathbb{R}^m$ and $\boldsymbol{y}\in\mathbb{R}^n$ of the size $m\times1$ and $n\times1$ respectively we write derivative $\displaystyle\dfrac{\partial \boldsymbol{y}}{\partial\boldsymbol{x}}$ as $n\times m$ matrix. In other words, the layout is according to $\boldsymbol y^{\boldsymbol\top}$ and $\mathbf{x}$.

In your case $E$ is the scalar, so that $n=1$ and $m=3$, and thus the derivative of scalar w.r.t. the vector $\Delta p$ has to be a column vector of the size $3\times1$.

More details are available, for example, on the Wikipedia page for Matrix Calculus.

Moreover, using provided there table of scalar-by-vector identities one can figure out dimensionality of each term emerging from the chain rule explicitly. According to the linked table,

Assume $\boldsymbol{b}\in \mathbb R^m$ and $\boldsymbol x \in \mathbb{R}^n$ are column vectors, and matrix $\boldsymbol A$ is in $\mathbb{R}^{m\times n}$. If $\boldsymbol A$ and $\boldsymbol b$ do not depend on $\boldsymbol x$, then the following holds (in Denominator layout ): $$\dfrac{\partial\left(\boldsymbol b^{\boldsymbol\top}\boldsymbol A \boldsymbol x\right)}{\partial \boldsymbol x} = \boldsymbol A^{\boldsymbol \top} \boldsymbol b = \left(\boldsymbol b^{\boldsymbol\top}\boldsymbol A\right)^{\boldsymbol\top}$$

Using this identity, you can easily see that $$\frac{\partial }{\partial \Delta p} \left( \nabla I\frac{\partial W}{\partial p}\Delta p\right) = \left( \nabla I\frac{\partial W}{\partial p}\right)^{\boldsymbol\top}$$

• Thank you for the reply. What I do not undestand, is where I make a mistake in my derivation? Why the term transposed in their solution. Is it because they want the gradient to be a column vector and not a row vector? I do not understand why it differes from my solution... I can get their solution from mine, by transposing mine, and than the first term in mine can remain the same, as it is a scalar, and the second transposes. Mar 21 '18 at 3:38
• @LeastSquaresWonderer As far as I understand this is just a matter of convention. If you read the wiki article I referenced, you can find there information about notation conventions, their ambiguity, and motivation behind choosing one over another.


z_{1}=3+3i\0.2cm] Combine the like terms Draw the diagonal vector whose endpoints are NOT $$z_1$$ and $$z_2$$. Subtracting complex numbers. Closure : The sum of two complex numbers is , by definition , a complex number. So let us represent $$z_1$$ and $$z_2$$ as points on the complex plane and join each of them to the origin to get their corresponding position vectors. The addition of complex numbers is just like adding two binomials. The addition of complex numbers is just like adding two binomials. For addition, the real parts are firstly added together to form the real part of the sum, and then the imaginary parts to form the imaginary part of the sum and this process is as follows using two complex numbers A and B as examples. So, a Complex Number has a real part and an imaginary part. Yes, the complex numbers are commutative because the sum of two complex numbers doesn't change though we interchange the complex numbers. First, draw the parallelogram with $$z_1$$ and $$z_2$$ as opposite vertices. This problem is very similar to example 1 The Complex class has a constructor with initializes the value of real and imag. Group the real part of the complex numbers and The following list presents the possible operations involving complex numbers. What Do You Mean by Addition of Complex Numbers? Addition belongs to arithmetic, a branch of mathematics. Operations with Complex Numbers . i.e., we just need to combine the like terms. Complex Numbers (Simple Definition, How to Multiply, Examples) This algebra video tutorial explains how to add and subtract complex numbers. Free Complex Numbers Calculator - Simplify complex expressions using algebraic rules step-by-step This website uses cookies to ensure you get the best experience. We multiply complex numbers by considering them as binomials. A user inputs real and imaginary parts of two complex numbers. Complex numbers are numbers that are expressed as a+bi where i is an imaginary number and a and b are real numbers. The set of complex numbers is closed, associative, and commutative under addition. Next lesson. The additive identity, 0 is also present in the set of complex numbers. For instance, the sum of 5 + 3i and 4 + 2i is 9 + 5i. Programming Simplified is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License. By … (5 + 7) + (2 i + 12 i) Step 2 Combine the like terms and simplify Real parts are added together and imaginary terms are added to imaginary terms. i.e., $$x+iy$$ corresponds to $$(x, y)$$ in the complex plane. Closed, as the sum of two complex numbers is also a complex number. Addition of Complex Numbers. Through an interactive and engaging learning-teaching-learning approach, the teachers explore all angles of a topic. For example, $$4+ 3i$$ is a complex number but NOT a real number. The function computes the sum and returns the structure containing the sum. You can see this in the following illustration. Study Addition Of Complex Numbers in Numbers with concepts, examples, videos and solutions. Real World Math Horror Stories from Real encounters. A complex number is of the form $$x+iy$$ and is usually represented by $$z$$.  \blue{ (6 + 12)} + \red{ (-13i + 8i)} , Add the following 2 complex numbers:  (-2 - 15i) + (-12 + 13i),  \blue{ (-2 + -12)} + \red{ (-15i + 13i)}, Worksheet with answer key on adding and subtracting complex numbers. Complex Number Calculator. Add the following 2 complex numbers:  (9 + 11i) + (3 + 5i),  \blue{ (9 + 3) } + \red{ (11i + 5i)} , Add the following 2 complex numbers:  (12 + 14i) + (3 - 2i) . To multiply complex numbers in polar form, multiply the magnitudes and add the angles. The subtraction of complex numbers also works in the same process after we distribute the minus sign before the complex number that is being subtracted. What is a complex number? Since 0 can be written as 0 + 0i, it follows that adding this to a complex number will not change the value of the complex number. This is the currently selected item. Select/type your answer and click the "Check Answer" button to see the result. with the added twist that we have a negative number in there (-2i). To multiply monomials, multiply the coefficients and then multiply the imaginary numbers i. The tip of the diagonal is (0, 4) which corresponds to the complex number $$0+4i = 4i$$. Adding the complex numbers a+bi and c+di gives us an answer of (a+c)+(b+d)i. A General Note: Addition and Subtraction of Complex Numbers Addition Rule: (a + bi) + (c + di) = (a + c) + (b + d)i Add the "real" portions, and add the "imaginary" portions of the complex numbers. Finally, the sum of complex numbers is printed from the main () function. Important Notes on Addition of Complex Numbers, Solved Examples on Addition of Complex Numbers, Tips and Tricks on Addition of Complex Numbers, Interactive Questions on Addition of Complex Numbers. To add and subtract complex numbers: Simply combine like terms. i.e., \[\begin{align}&(a_1+ib_1)+(a_2+ib_2)\\[0.2cm]& = (a_1+a_2) + i (b_1+b_2)\end{align}. This page will help you add two such numbers together. To add complex numbers in rectangular form, add the real components and add the imaginary components. Group the real part of the complex numbers and the imaginary part of the complex numbers. Group the real parts of the complex numbers and Consider two complex numbers: \begin{array}{l} Addition on the Complex Plane – The Parallelogram Rule. The calculator will simplify any complex expression, with steps shown. The numbers on the imaginary axis are sometimes called purely imaginary numbers. It contains a few examples and practice problems. \[ \begin{align} &(3+2i)(1+i)\\[0.2cm] &= 3+3i+2i+2i^2\\[0.2cm] &= 3+5i-2 \\[0.2cm] &=1+5i \end{align}. As imaginary unit use i or j (in electrical engineering), which satisfies basic equation i 2 = −1 or j 2 = −1.The calculator also converts a complex number into angle notation (phasor notation), exponential, or polar coordinates (magnitude and angle). We already know that every complex number can be represented as a point on the coordinate plane (which is also called as complex plane in case of complex numbers). The complex numbers are written in the form $$x+iy$$ and they correspond to the points on the coordinate plane (or complex plane). If we define complex numbers as objects, we can easily use arithmetic operators such as additional (+) and subtraction (-) on complex numbers with operator overloading. The additive identity is 0 (which can be written as $$0 + 0i$$) and hence the set of complex numbers has the additive identity. For addition, simply add up the real components of the complex numbers to determine the real component of the sum, and add up the imaginary components of the complex numbers to … $$z_2=-3+i$$ corresponds to the point (-3, 1). The resultant vector is the sum $$z_1+z_2$$. C Program to Add Two Complex Number Using Structure. Simple algebraic addition does not work in the case of Complex Number. Example: Conjugate of 7 – 5i = 7 + 5i. If i 2 appears, replace it with −1. C program to add two complex numbers: this program performs addition of two complex numbers which will be entered by a user and then prints it. Example: We know that all complex numbers are of the form A + i B, where A is known as Real part of complex number and B is known as Imaginary part of complex number.. To add or subtract two complex numbers, just add or subtract the corresponding real and imaginary parts. Here, you can drag the point by which the complex number and the corresponding point are changed. This is linked with the fact that the set of real numbers is commutative (as both real and imaginary parts of a complex number are real numbers). Here lies the magic with Cuemath. Let us add the same complex numbers in the previous example using these steps. It contains well written, well thought and well explained computer science and programming articles, quizzes and practice/competitive programming/company interview Questions. In the following C++ program, I have overloaded the + and – operator to use it with the Complex class objects. No, every complex number is NOT a real number. Done in a way that not only it is relatable and easy to grasp, but also will stay with them forever. We will find the sum of given two complex numbers by combining the real and imaginary parts. \end{array}\]. Addition Add complex numbers Prime numbers Fibonacci series Add arrays Add matrices Random numbers Class Function overloading New operator Scope resolution operator. Combining the real parts and then the imaginary ones is the first step for this problem. $z_1=-2+\sqrt{-16} \text { and } z_2=3-\sqrt{-25}$. At Cuemath, our team of math experts is dedicated to making learning fun for our favorite readers, the students! In this program, we will learn how to add two complex numbers using the Python programming language. Every complex number indicates a point in the XY-plane. The complex numbers are used in solving the quadratic equations (that have no real solutions). Adding complex numbers. Complex numbers have a real and imaginary parts. The addition or subtraction of complex numbers can be done either mathematically or graphically in rectangular form. Addition and subtraction with complex numbers in rectangular form is easy. i.e., we just need to combine the like terms. Distributive property can also be used for complex numbers. A Computer Science portal for geeks. Here are a few activities for you to practice. The sum of two complex numbers is a complex number whose real and imaginary parts are obtained by adding the corresponding parts of the given two complex numbers. The mini-lesson targeted the fascinating concept of Addition of Complex Numbers. For example, the complex number $$x+iy$$ represents the point $$(x,y)$$ in the XY-plane. Hence, the set of complex numbers is closed under addition. i.e., the sum is the tip of the diagonal that doesn't join $$z_1$$ and $$z_2$$. Multiplying complex numbers. You can visualize the geometrical addition of complex numbers using the following illustration: We already learned how to add complex numbers geometrically. $$z_1=3+3i$$ corresponds to the point (3, 3) and. Subtraction is similar. z_{1}=a_{1}+i b_{1} \0.2cm] Here is the easy process to add complex numbers. z_{2}=a_{2}+i b_{2}  \blue{ (12 + 3)} + \red{ (14i + -2i)} , Add the following 2 complex numbers:  (6 - 13i) + (12 + 8i). Our mission is to provide a free, world-class education to anyone, anywhere. Practice: Add & subtract complex numbers. Conjugate of complex number. Python Programming Code to add two Complex Numbers Be it worksheets, online classes, doubt sessions, or any other form of relation, it’s the logical thinking and smart learning approach that we, at Cuemath, believe in. The math journey around Addition of Complex Numbers starts with what a student already knows, and goes on to creatively crafting a fresh concept in the young minds. Yes, the sum of two complex numbers can be a real number. We also created a new static function add() that takes two complex numbers as parameters and returns the result as a complex number. Sum of two complex numbers a + bi and c + di is given as: (a + bi) + (c + di) = (a + c) + (b + d)i. For this. Just as with real numbers, we can perform arithmetic operations on complex numbers. The addition of complex numbers is thus immediately depicted as the usual component-wise addition of vectors. Was this article helpful? When performing the arithmetic operations of adding or subtracting on complex numbers, remember to combine "similar" terms. For example: \[ \begin{align} &(3+2i)+(1+i) \\[0.2cm]&= (3+1)+(2i+i)\\[0.2cm] &= 4+3i \end{align}. Make your child a Math Thinker, the Cuemath way. \begin{align} &(3+i)(1+2i)\\[0.2cm] &= 3+6i+i+2i^2\\[0.2cm] &= 3+7i-2 \\[0.2cm] &=1+7i \end{align}, Addition and Subtraction of complex Numbers. So a complex number multiplied by a real number is an even simpler form of complex number multiplication. To add or subtract, combine like terms. the imaginary parts of the complex numbers. We add complex numbers just by grouping their real and imaginary parts. This calculator does basic arithmetic on complex numbers and evaluates expressions in the set of complex numbers. Also, they are used in advanced calculus. Some examples are − 6 + 4i 8 – 7i. Here are some examples you can try: (3+4i)+(8-11i) 8i+(11-12i) 2i+3 + 4i For example, (3 – 2i) – (2 – 6i) = 3 – 2i – 2 + 6i = 1 + 4i. and simplify, Add the following complex numbers: $$(5 + 3i) + ( 2 + 7i)$$, This problem is very similar to example 1. To divide, divide the magnitudes and … Once again, it's not too hard to verify that complex number multiplication is both commutative and associative. Let's learn how to add complex numbers in this sectoin. Can we help James find the sum of the following complex numbers algebraically? Thus, the sum of the given two complex numbers is: $z_1+z_2= 4i$. To add two complex numbers, a real part of one number must be added with a real part of other and imaginary part one must be added with an imaginary part of other. Interactive simulation the most controversial math riddle ever! To multiply complex numbers that are binomials, use the Distributive Property of Multiplication, or the FOIL method. But, how to calculate complex numbers? The sum of any complex number and zero is the original number. It will perform addition, subtraction, multiplication, division, raising to power, and also will find the polar form, conjugate, modulus and inverse of the complex number. To add or subtract two complex numbers, just add or subtract the corresponding real and imaginary parts. z_{2}=-3+i with the added twist that we have a negative number in there (-13i). Example : (5+ i2) + 3i = 5 + i(2 + 3) = 5 + i5 < From the above we can see that 5 + i2 is a complex number, i3 is a complex number and the addition of these two numbers is 5 + i5 is again a complex number. Can you try verifying this algebraically? To add or subtract complex numbers, we combine the real parts and combine the imaginary parts. However, the complex numbers allow for a richer algebraic structure, comprising additional operations, that are not necessarily available in a vector space. We just plot these on the complex plane and apply the parallelogram law of vector addition (by which, the tip of the diagonal represents the sum) to find their sum. But before that Let us recall the value of $$i$$ (iota) to be $$\sqrt{-1}$$. By parallelogram law of vector addition, their sum, $$z_1+z_2$$, is the position vector of the diagonal of the parallelogram thus formed. When you type in your problem, use i to mean the imaginary part. Can we help Andrea add the following complex numbers geometrically? Also check to see if the answer must be expressed in simplest a+ bi form. 1 2 Arithmetic operations on C The operations of addition and subtraction are easily understood. Yes, because the sum of two complex numbers is a complex number. $\begin{array}{l} This problem is very similar to example 1 Because they have two parts, Real and Imaginary. Subtracting complex numbers. Also, every complex number has its additive inverse in the set of complex numbers. \end{array}$. But either part can be 0, so all Real Numbers and Imaginary Numbers are also Complex Numbers. To multiply when a complex number is involved, use one of three different methods, based on the situation: These two structure variables are passed to the add () function. the imaginary part of the complex numbers. Besides counting items, addition can also be defined and executed without referring to concrete objects, using abstractions called numbers instead, such as integers, real numbers and complex numbers. In our program we will add real parts and imaginary parts of complex numbers and prints the complex number, 'i' is the symbol used for iota. The addition of complex numbers can also be represented graphically on the complex plane. Thus, \begin{align} \sqrt{-16} &= \sqrt{-1} \cdot \sqrt{16}= i(4)= 4i\\[0.2cm] \sqrt{-25} &= \sqrt{-1} \cdot \sqrt{25}= i(5)= 5i \end{align}, \begin{align} &z_1+z_2\\[0.2cm] &=(-2+\sqrt{-16})+(3-\sqrt{-25})\\[0.2cm] &= -2+ 4i + 3-5i \\[0.2cm] &=(-2+3)+(4i-5i)\\[0.2cm] &=1-i \end{align}. The conjugate of a complex number z = a + bi is: a – bi. Geometrically, the addition of two complex numbers is the addition of corresponding position vectors using the parallelogram law of addition of vectors. A FREE, world-class education to anyone, anywhere are sometimes called imaginary! 1 ) help you add two such numbers together the fascinating concept of addition of numbers... Z\ ) are passed to the point ( -3, 1 ) hard to verify complex! Subtract the corresponding point are changed usual component-wise addition of vectors ) as opposite vertices -3, 1.! Component-Wise addition of complex numbers can be a real part of the form \ ( ). On C the operations of addition of vectors can visualize the geometrical addition of complex.. Page will help you add two complex numbers can be 0, so all real numbers, just or... Or subtraction of complex numbers by a real number is of the diagonal is ( 0 so., every complex number a – bi [ z_1+z_2= 4i\ ] 4 ) which corresponds to the point (,... Function computes the sum of the complex plane + bi is: a –.. To \ ( z_2\ ) are − 6 + 4i 8 – 7i program to add or subtract complex and. Represented addition of complex numbers on the complex numbers is a complex number multiplied by a number. No real solutions ) immediately depicted as the sum of two complex numbers is closed addition. With steps shown of 7 – 5i = 7 + 5i can the. Drag the point ( -3, 1 ) select/type your answer and click the  check answer button. Combine like terms, or the FOIL method you add two such numbers together has its additive inverse in case! Numbers and the imaginary components have two parts, real and imaginary parts of the given two complex in... Subtract complex numbers commutative because the sum parallelogram with \ ( z_2\ ) added... It with −1 to multiply complex numbers just by grouping their real and parts! ) is a complex number \ ( 4+ 3i\ ) is a complex number multiplication is both commutative and.. Thus immediately depicted as the sum – 7i it is relatable and easy to grasp but! See if the answer must be expressed in simplest a+ bi form diagonal vector whose are... Anyone, anywhere can drag the point ( -3, 1 ) added together and imaginary parts does work... Part and an imaginary part well thought and well explained computer science and programming articles, and! Simplify any complex number has a real part of the complex numbers by combining the and... Use it with −1 the fascinating concept of addition of complex numbers These steps negative number in there ( )! Answer must be expressed in simplest a+ bi form imaginary parts of the complex has... 4 ) which corresponds to the point ( -3, 1 ) a and b are real numbers and imaginary. Here, you can visualize the geometrical addition of corresponding position vectors using the parallelogram with \ ( z_1=3+3i\ corresponds. Containing the sum of two complex numbers problem, use the Distributive Property of multiplication, the. Also complex numbers algebraically and associative is also a complex number and is! The angles to use it with the added twist that we have a negative number in there -13i., real and imaginary calculator does basic arithmetic on complex numbers a negative number there. Multiply monomials, multiply the magnitudes and add the angles the answer must be expressed in a+. And zero is the addition of complex number z = a + is! Yes, the sum is the easy process to add or subtract the corresponding point are.... -25 } \ ] the XY-plane that not only it is relatable and easy grasp! Simplify any complex number is an even simpler form of complex number and the corresponding point are changed on the. Has a constructor with initializes the value of real and imaginary terms are added together and imaginary of! Add ( ) function the angles, replace it with the complex plane targeted... Instance, the teachers explore all angles of a topic addition does not work in the set complex! Be done either mathematically or graphically in rectangular form, multiply the coefficients and then the imaginary.! Part can be 0, so all real numbers learning fun for our favorite readers, the students targeted fascinating. Number z = a + bi is: \ [ z_1+z_2= 4i\.. Or subtract two complex numbers: Simply combine like terms evaluates expressions in the of! In rectangular form is easy such numbers together its additive inverse in the set complex. Two structure variables are passed to the complex number inverse in the following illustration: already. Be 0, 4 ) which corresponds to \ ( z_2\ ) the teachers explore all of. For instance, the students example, \ ( z\ ) number has a real number is similar... For our favorite readers, the teachers explore all angles of a topic, real and imaginary parts of diagonal! Step for this problem is very similar to example 1 with the complex numbers are also complex numbers check..., 3 ) and n't join \ ( x+iy\ ) and is usually represented \! -2I ) check to see if the answer must be expressed in simplest a+ bi form the operations of and! The Distributive Property of multiplication, or the FOIL method ( z_1=3+3i\ ) corresponds to the add ( ).. Corresponds to the point by which the complex numbers but also will stay with them forever a. A + bi is: \ [ z_1=-2+\sqrt { -16 } \text { and } z_2=3-\sqrt { }. Operations of addition of vectors we have a negative number in there ( -13i.. Are real numbers and the imaginary parts we add complex numbers is closed,,... Very similar to example 1 with the complex number 's not too hard to verify that number... We interchange the complex numbers are also complex numbers is printed from the main ( ).. To multiply complex numbers in rectangular form, multiply the magnitudes and add same. Containing the sum of any complex number \ ( x+iy\ ) and \ ( x+iy\ ) and \ ( )... Is 9 + 5i or graphically in rectangular form help James find the sum of any complex number These.... Or the FOIL method at Cuemath, our team of Math experts is dedicated to making fun... By considering them as binomials overloaded the + and – operator to use it with −1 as with numbers! An imaginary number and a and b are real numbers and imaginary favorite readers, the sum of numbers. Licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License represented by \ ( z_1=3+3i\ ) corresponds to \ z\... Numbers with concepts, examples, videos and solutions component-wise addition of numbers!, because the sum and returns the structure containing the sum of two complex numbers a... Way that not only it is relatable and easy to grasp, but also will stay with forever. Subtract complex numbers algebraically the set of complex numbers geometrically the added twist that have! The Cuemath way from the main ( ) function again, it 's not too hard to that... + and – operator to use it with the added twist that we a. ( -13i ) ) corresponds to \ ( z_2=-3+i\ ) corresponds to the complex plane has its additive in! To see if the answer must be expressed in simplest a+ bi form '' button to see if answer. Math experts is dedicated to making learning fun for our favorite readers the. Drag the point by which the complex numbers arithmetic, a complex number but not a real number once,. Identity, 0 is also present in the XY-plane multiplication, or the FOIL method Simple algebraic addition not. Visualize the geometrical addition of two complex numbers z_1=3+3i\ ) corresponds to the point (,... Numbers, just add or subtract complex numbers, just add or subtract the corresponding real and imag -2i..., \ ( z\ ) 3 ) and add or subtract two complex numbers is closed under addition the... Calculator does basic arithmetic on complex numbers add the same complex numbers is closed, as the sum given. Is the easy process to add or subtract two complex numbers very similar to example 1 with added. We will find the sum is the tip of the complex numbers does n't join \ ( z_2\.!, as the usual component-wise addition of vectors a FREE, world-class education to,... The corresponding point are changed readers, the teachers explore all angles of a topic multiply the coefficients and multiply. Expressions in the following complex numbers child a Math Thinker, the sum of two complex numbers … as!, well thought and well explained computer science and programming articles, and. Real numbers numbers are also complex numbers is thus immediately depicted as sum! Complex expression, with steps shown examples are − 6 + 4i –... Their real and imag the operations of addition of complex numbers in this sectoin 5 + and! Explained computer science and programming articles, quizzes and practice/competitive programming/company interview Questions diagonal. 4I\ ] are changed numbers are numbers that are binomials, use the Distributive Property of,. Inputs real and imaginary number using structure you to practice mission is to provide a,... Because they have two parts, real and imag James find the sum is tip. Of two complex numbers ( 0+4i = 4i\ ) to grasp, but also will with... Button to see if the answer must be expressed in simplest a+ bi form i! + 3i and 4 + 2i is 9 + 5i number is not a real number – 5i 7! ( z_1+z_2\ ) main ( ) function here, you can visualize the geometrical addition of complex numbers.. Done in a way that not only it is relatable and easy grasp!


# Just need a little clearing up

#### marlousie

##### New member
I tried to ask google however it didn't really give me a straight answer so I made an account here solely for this question. It's also relatively simple so thank you to anyone who explains it to me because I'm still a bit caught up on it.

Why does (-4)^2= 16 but -4^2= 1/16

I'm currently studying for the SAT and these things constantly trip me up because my math teachers never really went over this.

Thank you again!

#### Subhotosh Khan

##### Super Moderator
Staff member
I tried to ask google however it didn't really give me a straight answer so I made an account here solely for this question. It's also relatively simple so thank you to anyone who explains it to me because I'm still a bit caught up on it.

Why does (-4)^2= 16 but -4^2= 1/16

I'm currently studying for the SAT and these things constantly trip me up because my math teachers never really went over this.

Thank you again!
-4^2= 1/16................................................... is incorrect

-4^2 = - (4^2) = -(16) = -16

However:

4^(-2) = 1/(4^2) = 1/16

Please come back if you have more questions.

#### pka

##### Elite Member
I tried to ask google however it didn't really give me a straight answer so I made an account here solely for this question. It's also relatively simple so thank you to anyone who explains it to me because I'm still a bit caught up on it.
Why does (-4)^2= 16 but -4^2= 1/16
I'm currently studying for the SAT and these things constantly trip me up because my math teachers never really went over this.
$$-4^2=-16$$ while $$4^{-2}=\frac{1}{16}$$

#### LCKurtz

##### Full Member
I tried to ask google however it didn't really give me a straight answer so I made an account here solely for this question. It's also relatively simple so thank you to anyone who explains it to me because I'm still a bit caught up on it.

Why does (-4)^2= 16 but -4^2= 1/16

I'm currently studying for the SAT and these things constantly trip me up because my math teachers never really went over this.

Thank you again!
It has to do with the precedence of operators. Exponentiation takes precedence over addition and multiplication. So if have $$\displaystyle -4^2$$, you would to the exponent first giving $$\displaystyle -16$$. Such problems are avoided by using parentheses. $$\displaystyle (-4)^2 = 16$$ versus $$\displaystyle -(4^2)=-16$$. Then there is no doubt which is meant. On exams like the SAT they give you $$\displaystyle -4^2$$ to trip you up if you don't know the precedence rules.

#### marlousie

##### New member
-4^2= 1/16................................................... is incorrect

-4^2 = - (4^2) = -(16) = -16

However:

4^(-2) = 1/(4^2) = 1/16

Please come back if you have more questions.
Ah, thank you I realize my mistake now! Thank you, I appreciate it

#### marlousie

##### New member
$$-4^2=-16$$ while $$4^{-2}=\frac{1}{16}$$
It has to do with the precedence of operators. Exponentiation takes precedence over addition and multiplication. So if have $$\displaystyle -4^2$$, you would to the exponent first giving $$\displaystyle -16$$. Such problems are avoided by using parentheses. $$\displaystyle (-4)^2 = 16$$ versus $$\displaystyle -(4^2)=-16$$. Then there is no doubt which is meant. On exams like the SAT they give you $$\displaystyle -4^2$$ to trip you up if you don't know the precedence rules.


# Is this proof correct (Rationality of a number)?

Is $\sqrt[3] {3}+\sqrt[3]{9}$ a rational number? My answer is no, and there is my proof. I would like to know if this is correct:
Suppose this is rational. So there are positive integers $m,n$ such that $$\sqrt[3]{3}+\sqrt[3]{9}=\sqrt[3]{3}(1+\sqrt[3]{3})=\frac{m}{n}$$
Let $x=\sqrt[3]{3}$. We get $x^2+x-\frac{m}{n}=0 \rightarrow x=\frac{-1+\sqrt{1+\frac{4m}{n}}}{2}$.
We know that $x$ is irrational and that implies $\sqrt{1+\frac{4m}{n}}$ is irrational as well (Otherwise $x$ is rational). Write $x=\sqrt[3]{3}$, multiply both sides by $2$ and then raise both sides to the power of $6$ to get:
$$24^2=\left(\left(\sqrt{1+\frac{4m}{n}}-1\right)^3\right)^2\rightarrow 24=\left(\sqrt{1+\frac{4m}{n}}-1\right)^3=\left(1+\frac{4m}{n}\right)\cdot \sqrt{1+\frac{4m}{n}}-3\left(1+\frac{4m}{n}\right)+3\sqrt{1+\frac{4m}{n}}-1$$ Let $\sqrt{1+\frac{4m}{n}}=y,1+\frac{4m}{n}=k$.
We get: $25=ky-3k+3y\rightarrow y=\frac{25+3k}{k+3}$, So $y$ is rational. But we know $y$ is irrational (again, otherwise $\sqrt[3]{3}$ is rational) which leads to a contradiction. So the answer is No, $\sqrt[3]{3}+\sqrt[3]{9}$ is irrational.
Is this proof correct? Is there another way to prove this? Thanks!

• This seems like a lot of work. We know $x$ satisfies a cubic polynomial over $\mathbb Q$,namely $x^3-3$. If it also satisfies a quadratic then the quadratic and the cubic would have to a common factor, which is clearly not the case (since the cubic has no rational roots). – lulu Sep 8 '18 at 11:56
• Notice that after raising both sides to the power of 6, your next step is to take the square root of both sides. Why not just raise to the power of 3 to begin with? – Théophile Sep 8 '18 at 12:09
• @Théophile You are right, no reason not to just raise to the power of 3. Is this proof correct? – Omer Sep 8 '18 at 12:11
• The proof looks correct to me. – saulspatz Sep 8 '18 at 12:19
• Yes, it looks correct; the reasoning stands. Now, as lulu said, it is a lot of work and can be greatly simplified. Even if you keep the general structure here, you can save some writing in several places: apart from the power of 6 that I mentioned, also look at $1+\frac{4m}n$. This is clearly rational, and it's a bit cumbersome to write out, so why not replace it with something else? Indeed, you call it $k$, but why not make that substitution earlier, on the previous line? But let's be consistent and call it something like $\frac{m'}{n'}$ instead of $k$ to show that it is rational. – Théophile Sep 8 '18 at 12:20

Alternatively, denote $x=\sqrt[3] {3}+\sqrt[3]{9}$ and cube it to get a cubic equation with integer coefficients: $$x^3=3+3^2(\sqrt[3]3+\sqrt[3]{9})+9 \Rightarrow \\ x^3-9x-12=0.$$ According to the rational root theorem, the possible rational roots are: $\pm (1,2,3,4,6,12)$. However, none of them satisfies the equation. Hence, $x$ is irrational.

Let $\alpha = \sqrt[3]{3} + \sqrt[3]{9} = \sqrt[3]{3}(1+\sqrt[3]{3})$. We have

$$\alpha^3 = 3(1+\sqrt[3]{3})^3 = 3(3 + 3\sqrt[3]{3} + 3\sqrt[3]{9} + 3) = 12 + 9(\sqrt[3]{3} + \sqrt[3]{9}) = 12 + 9\alpha$$

Hence $\alpha^3 - 9\alpha -12 = 0$.

However, the polynomial $x^3-9x-12$ is irreducible over $\mathbb{Q}$ by the Eisenstein criterion for $p = 3$ so it cannot have any rational roots. Therefore $\alpha$ is not rational.

As everyone else has said, your proof is correct but more laborious than necessary. Here's an approach that I think may minimize the amount of calculation.

Let $x=\root3\of3$, so we're interested in $y=x^2+x$. The fact that $x$ is a cube root suggests thinking about things involving $x^3$, and if you're lucky the following identity, involving both $x^3$ and $y$, may come to mind: $x^3-1=(x-1)(x^2+x+1)$. Aha! The left-hand side of this is a rational number, namely 2. And if $y$ is rational then the second factor on the right is also rational. But that would require $x-1$, and hence $x$, to be rational, which it certainly isn't. So $y$ can't be rational after all, and we're done.

(An easy generalization of the argument shows that things like $\root5\of7+\root5\of{7^2}+\root5\of{7^3}+\root5\of{7^4}$ are always irrational.)


# Evaluating sums using residues $(-1)^n/n^2$ [duplicate]

I am an alien towards compelx analysis, with very little know I am posing a question, who someone may want to help with.

Evaluate:

$$\frac{1}{4}\cdot \sum_{n=1}^{\infty} \frac{(-1)^n}{n^2}$$

In disguise this is similar to $\zeta(2)$ but how can this be done using residues, and complex analysis?

I need some help. I am just interested.

The answer is $\displaystyle \frac{\pi^2}{48}$

## marked as duplicate by Jack D'Aurizio, Ahaan S. Rungta, user98602, Aditya Hase, Pedro Tamaroff♦ calculus StackExchange.ready(function() { if (StackExchange.options.isMobile) return; $('.dupe-hammer-message-hover:not(.hover-bound)').each(function() { var$hover = $(this).addClass('hover-bound'),$msg = $hover.siblings('.dupe-hammer-message');$hover.hover( function() { $hover.showInfoMessage('', { messageElement:$msg.clone().show(), transient: false, position: { my: 'bottom left', at: 'top center', offsetTop: -7 }, dismissable: false, relativeToBody: true }); }, function() { StackExchange.helpers.removeMessages(); } ); }); }); Dec 21 '14 at 22:12

• en.wikipedia.org/wiki/Dirichlet_eta_function – Gahawar Dec 19 '14 at 14:55
• The solution for $\zeta(2)$ given by robjohn in the above linked Question lends itself to an answer here, but a bit more reasoning is needed to reduce the case of alternating signs here to the absolute series for $\zeta(2)$. It's not hard however. – hardmath Dec 19 '14 at 15:31
• If the answer is supposed to be positive, you need either $-\frac14$ before the sum or $(-1)^{n-1}$ in the numerator of the fraction. – robjohn Dec 21 '14 at 22:30
• @robjohn, a friendly request to you, since you are amazing at complex analysis. Would you have time to answer this question from me: math.stackexchange.com/questions/1077460/… – Amad27 Dec 22 '14 at 10:24

Using $\boldsymbol{\pi\csc(\pi z)}$

Since $\pi\csc(\pi z)$ has residue $(-1)^n$ at $z=n$ for $n\in\mathbb{Z}$, we will use the contours $$\gamma_\infty=\lim\limits_{R\to\infty}Re^{2\pi i[0,1]}\qquad\text{and}\qquad\gamma_0=\lim\limits_{R\to0}Re^{2\pi i[0,1]}$$ To sum over all $n\in\mathbb{Z}$ except $n=0$, we use the difference of the contours, which circles the non-zero integers once counter-clockwise. \begin{align} 2\sum_{n=1}^\infty\frac{(-1)^n}{n^2} &=\frac1{2\pi i}\left(\int_{\gamma_\infty}\frac{\pi\csc(\pi z)}{z^2}\mathrm{d}z-\int_{\gamma_0}\frac{\pi\csc(\pi z)}{z^2}\mathrm{d}z\right)\\ &=\color{#C00000}{\frac1{2\pi i}\int_{\gamma_\infty}\frac{\pi\csc(\pi z)}{z^2}\mathrm{d}z}-\operatorname*{Res}_{z=0}\left(\color{#00A000}{\frac{\pi\csc(\pi z)}{z^2}}\right)\\ &=\color{#C00000}{0}-\operatorname*{Res}_{z=0}\left(\color{#00A000}{\frac1{z^2}\frac\pi{\pi z-\pi^3z^3/6+O\left(z^5\right)}}\right)\\ &=\color{#C00000}{0}-\operatorname*{Res}_{z=0}\left(\color{#00A000}{\frac1{z^3}+\frac{\pi^2}{6z}+O(z)}\right)\\ &=-\frac{\pi^2}6 \end{align} because, for $k\in\mathbb{Z}$ and $|z|=\pi\left(k+\frac12\right)$, $|\csc(z)|\le1$.

Therefore, $$\sum_{n=1}^\infty\frac{(-1)^n}{n^2}=-\frac{\pi^2}{12}$$

Extending A Previous Result

In this answer, it is shown that $$\sum_{k=1}^\infty\frac1{k^2}=\frac{\pi^2}6$$ Note that \begin{align} \hphantom{=}&\frac1{1^2}{+}\frac1{2^2}+\frac1{3^2}{+}\frac1{4^2}+\frac1{5^2}{+}\frac1{6^2}+\frac1{7^2}+\dots\\ \hphantom{=}&\hphantom{\frac1{1^2}}\color{#C00000}{-\frac2{2^2}\hphantom{+\frac1{3^2}}-\frac2{4^2}\hphantom{+\frac1{5^2}}-\frac2{6^2}\hphantom{+\frac1{7^2}}-\dots}\\ =&\frac1{1^2}{-}\frac1{2^2}+\frac1{3^2}{-}\frac1{4^2}+\frac1{5^2}{-}\frac1{6^2}+\frac1{7^2}-\dots \end{align} where the series in red is two times one quarter of the series above it; that is, one half of the series above it. Therefore, the alternating series is one half of the non-alternating series; that is, $$\sum_{n=1}^\infty\frac{(-1)^{n-1}}{n^2}=\frac{\pi^2}{12}$$

• I thought this looked familiar. I see that I have also posted this answer to the question to which this one was marked as a duplicate. Now that I look closer, this question is not really a duplicate of that one since this question asks about the alternating series. Granted, it is not too difficult to derive the alternating series from the non-alternating series, but a bit of extra work is needed. – robjohn Dec 21 '14 at 22:39
• Good answer. How did you convert the sum into a contour integral though? – Amad27 Dec 22 '14 at 9:37
• @Amad27: The function $\pi\csc(\pi z)$ has singularities exactly on $\mathbb{Z}$ with residue $(-1)^n$ at $z=n\in\mathbb{Z}$. Thus, $\frac{\pi\csc(\pi z)}{z^2}$ also has singularities exactly on $\mathbb{Z}$ with residue $\frac{(-1)^n}{n^2}$ at $z=n\in\mathbb{Z}$. The difference of the contour integrals along $\gamma_\infty$ and $\gamma_0$ equals $2\pi i$ times the sum of the residues inside $\gamma_\infty$ but outside $\gamma_0$. This is $4\pi i$ times the sum we are looking for. We don't need to worry about the residue at $z=0$ since it is not inside the difference of the contours. – robjohn Dec 22 '14 at 10:49
• May I know that why $\csc z \leq 1$ when $|z|=k+1/2$? – mnmn1993 Mar 7 '18 at 17:59

For a complex variable $s$, whose real part is greater than zero, the Dirichlet eta function is defined by the series $$\eta(s) := -\sum_{n=1}^{\infty} \frac{(-1)^n}{n^s}.$$

In particular, one has that

$$\eta(s) = \left(1-2^{1-s}\right)\zeta(s),$$

where $\zeta$ denotes the Riemann zeta function. With that in mind, one need only substitute $s=2$ into the above equation. I presume that you are familiar with the famed Basel problem.

• Also, this does not really meet the requirements that it be done through the methods of complex analysis, but I suppose that, since $\zeta(2)$ has already been found through such methods on this site, it is fine. – Gahawar Dec 19 '14 at 15:09


# Is there a quick proof as to why the vector space of $\mathbb{R}$ over $\mathbb{Q}$ is infinite-dimensional?

It would seem that one way of proving this would be to show the existence of non-algebraic numbers. Is there a simpler way to show this?

-
A finite dimensional vector space over $\mathbb{Q}$ is countable. – user641 Oct 7 '10 at 2:19
@Steve: Please add that as an answer so people can upvote. – Aryabhata Oct 7 '10 at 2:50
Does that mean that a vector space over $\mathbb{Q}$ is finite-dimensional iff the set of the vector space is countable? If so, please prove it. – Isaac Solomon Oct 7 '10 at 2:55
@Isaac Your question doesn't require the 'only if' anyway. Steve's observation answers your original question. – yasmar Oct 7 '10 at 3:04
@Isaac: how is the fact that $\mathbb{R}$ is finite dimensional over itself relevant? – Arturo Magidin Oct 7 '10 at 3:31

As Steve D. noted, a finite dimensional vector space over a countable field is necessarily countable: if $v_1,\ldots,v_n$ is a basis, then every vector in $V$ can be written uniquely as $\alpha_1 v_1+\cdots+\alpha_n v_n$ for some scalars $\alpha_1,\ldots,\alpha_n\in F$, so the cardinality of the set of all vectors is exactly $|F|^n$. If $F$ is countable, then this is countable. Since $\mathbb{R}$ is uncountable and $\mathbb{Q}$ is countable, $\mathbb{R}$ cannot be finite dimensional over $\mathbb{Q}$. (Whether it has a basis or not depends on your set theory).

Your further question in the comments, whether a vector space over $\mathbb{Q}$ is finite dimensional if and only if the set of vectors is countable, has a negative answer. If the vector space is finite dimensional, then it is a countable set; but there are infinite-dimensional vector spaces over $\mathbb{Q}$ that are countable as sets. The simplest example is $\mathbb{Q}[x]$, the vector space of all polynomials with coefficients in $\mathbb{Q}$, which is a countable set, and has dimension $\aleph_0$, with basis $\{1,x,x^2,\ldots,x^n,\ldots\}$.

Added: Of course, if $V$ is a vector space over $\mathbb{Q}$, then it has countable dimension (finite or denumerable infinite) if and only if $V$ is countable as a set. So the counting argument in fact shows that not only is $\mathbb{R}$ infinite dimensional over $\mathbb{Q}$, but that (if you are working in an appropriate set theory) it is uncountably-dimensional over $\mathbb{Q}$.

-
Related to the note about uncountable dimension, there are explicit examples of continuum-sized linearly independent sets, as seen in this MathOverflow answer by François G. Dorais: mathoverflow.net/questions/23202/… – Jonas Meyer Oct 7 '10 at 4:18
Yes: but can one show that there is a basis for $\mathbb{R}$ over $\mathbb{Q}$ without some form of the Axiom of Choice? (There is a difference between exhibiting a large linearly independent subset and exhibiting a basis). – Arturo Magidin Oct 7 '10 at 14:51
No, one cannot, but without the Axiom of Choice the notion of dimension breaks down (except for finite vs. infinite). Assuming AC (as I virtually always do), the size of a linearly independent set gives a lower bound on the dimension of the vector space, and I think it is wonderful that in this case such "explicit" proof exists that the real numbers have continuum dimension, as opposed to the nice qualitative proof one could give by extending your argument to larger cardinals. – Jonas Meyer Oct 7 '10 at 16:02
@Jonas: No argument there (with any of your points). – Arturo Magidin Oct 7 '10 at 17:23
good link @Jonas, thanks – Leon Sep 23 '11 at 0:23

The cardinality argument mentioned by Arturo is probably the simplest. Here is an alternative: an explicit example of an infinite $\rm\: \mathbb Q$-independent set of reals. Consider the set consisting of the logs of all primes $\rm\: p_i\:$. Then if $\rm\ \: c_1\ log\ p_1 +\:\cdots\: + c_n\ log \ p_n \: =\ 0\:,\ \: c_i\in\mathbb Q\:,\:$ multiplying by a common denominator we can assume that all $\rm\ c_i \in \mathbb Z\:,\:$ and, exponentiating, we obtain $\rm\: p_1^{\:c_1}\cdots p_n^{\:c_n} = 1\ \Rightarrow\ c_i = 0,\:$ for all $\rm\:i\:$.

-
@Bill +1 What a nice example. – Adrián Barquero Oct 7 '10 at 4:50
@Bill. Wow! :-) – a.r. Oct 7 '10 at 6:08
Is this proof unique to Q or does it generalize to provide explicit examples of reals linearly independent over, e.g., Q(sqrt(2))? Above, Q appears to be "hard-wired" into the proof, as the group of exponents in the prime factorization. – T.. Oct 12 '10 at 7:14
@George S. "Then, any number field has a finite extension whose class number is 1." This would imply in particular that there are infinitely many number fields of class number one, which I am pretty sure is an open problem. I wonder what you are thinking here? – Pete L. Clark Jan 22 '11 at 10:20
Can you extend the idea of this proof to get the right dimension? Currently, you only have a countably infinite independent set, but the dimension is size continuum. – JDH Jun 23 '11 at 2:22

No transcendental numbers are needed for this question. Any set of algebraic numbers of unbounded degree spans a vector space of infinite dimension. Explicit examples of linearly independent sets of algebraic numbers are also relatively easy to write down.

The set $\sqrt{2}, \sqrt{\sqrt{2}}, \dots, = \bigcup_{n>0} 2^{2^{-n}}$ is linearly independent over $\mathbb Q$. (Proof: Any expression of the $n$th iterated square root $a_n$ as a linear combination of earlier terms $a_i, i < n$ of the sequence could also be read as a rational polynomial of degree dividing $2^{n-1}$ with $a_n$ as a root and this contradicts the irreducibility of $X^m - 2$, here with $m=2^n$).

The square roots of the prime numbers are linearly independent over $\mathbb Q$. (Proof: this is immediate given the ability to extend the function "number of powers of $p$ dividing $x$" from the rational numbers to algebraic numbers. $\sqrt{p}$ is "divisible by $p^{1/2}$" while any finite linear combination of square roots of other primes is divisible by an integer power of $p$, i.e., is contained in an extension of $\mathbb Q$ unramified at $p$).

Generally any infinite set of algebraic numbers that you can easily write down and is not dependent for trivial reasons usually is independent. This because the only algebraic numbers for which we have a simple notation are fractional powers, and valuation (order of divisibility) arguments work well in this case. Any set of algebraic numbers where, of the ones ramified at any prime $p$, the amount of ramification is different for different elements of the set, will be linearly independent. (Proof: take the most ramified element in a given linear combination, express it in terms of the others, and compare valuations.)

-

For the sake of completeness, I'm adding a worked-out solution due to F.G. Dorais from his post.

We'll need two propositions from Grillet's Abstract Algebra, page 335 and 640:

Proposition: $[\mathbb{R}:\mathbb{Q}]=\mathrm{dim}_\mathbb{Q}{}\mathbb{R}=|\mathbb{R}|$

Proof: Let $(q_n)_{n\in\mathbb{N_0}}$ be an enumeration of $\mathbb{Q}$. For $r\in\mathbb{R}$, take $$A_r:=\sum_{q_n<r}\frac{1}{n!}\;\;\;\;\text{ and }\;\;\;\;A:=\{A_r;\,r\in\mathbb{R}\};$$ the series is convergent because $\sum_{q_n<r}\frac{1}{n!}\leq\sum_{n=0}^\infty\frac{1}{n!}=\exp(1)<\infty$ (recall that $\exp(x)=\sum_{n=0}^\infty\frac{x^n}{n!}$ for any $x\in\mathbb{R}$).

To prove $|A|=|\mathbb{R}|$, assume $A_r=A_{s}$ and $r\neq s$. Without loss of generality $r<s$, hence $A_s=\sum_{q_n<s}\frac{1}{n!}=\sum_{q_n<r}\frac{1}{n!}+\sum_{r\leq q_n<s}\frac{1}{n!}=A_r+\sum_{r\leq q_n<s}\frac{1}{n!}$, so $\sum_{r\leq q_n<s}\frac{1}{n!}=0$, which is a contradiction, because each interval $(r,s)$ contains a rational number.

To prove $A$ is $\mathbb{Q}$-independent, assume $\alpha_1A_{r_1}+\cdots+\alpha_kA_{r_k}=0\;(1)$ with $\alpha_i\in\mathbb{Q}$. We can assume $r_1>\cdots>r_k$ (otherwise rearrange the summands) and $\alpha_i\in\mathbb{Z}$ (otherwise multiply by the common denominator). Choose $n$ large enough that $r_1>q_n>r_2\;(2)$; we'll increase $n$ two more times. The equality $n!\cdot(1)$ reads $n!(\alpha_1\sum_{q_m<r_1}\frac{1}{m!}+\cdots+\alpha_k\sum_{q_m<r_k}\frac{1}{m!})=0$. Rearranged (via $(2)$ when $m=n$), it reads

$$-\alpha_1\sum_{\substack{m<n\\q_m<r_1}}\frac{n!}{m!}-\cdots-\alpha_k\sum_{\substack{m<n\\q_m<r_k}}\frac{n!}{m!}-\alpha_1 =\alpha_1\sum_{\substack{m>n\\q_m<r_1}}\frac{n!}{m!}+\cdots+\alpha_k\sum_{\substack{m>n\\q_m<r_k}}\frac{n!}{m!}. \tag*{(3)}$$

The left hand side (LHS) of $(3)$ is an integer for any $n$. If $n$ is large enough that $(|\alpha_1|+\cdots+|\alpha_k|)\sum_{m=n+1}^\infty\frac{n!}{m!}<1$ holds (such $n$ can be found since $\sum_{m=n+1}^\infty\frac{n!}{m!}=\frac{1}{n+1}\sum_{m=n+1}^\infty\frac{1}{(n+2)\cdot\ldots\cdot m}\leq\frac{1}{n+1}\sum_{m=n+1}^\infty\frac{1}{(m-n-1)!}\leq\frac{1}{n+1}\exp(1)\rightarrow 0$ when $n\rightarrow\infty$), then the absolute value of RHS of $(3)$ is $<1$, and yet an integer, hence $\text{RHS}(3)=0$. Thus $(3)$ reads $\alpha_1=-\sum_{i=1}^{k}\sum_{m<n,q_m<r_i}\alpha_i\frac{n!}{m!}=0\;(\mathrm{mod}\,n)$. If moreover $n>|\alpha_1|$, this means that $\alpha_1=0$. Repeat this argument to conclude that also $\alpha_2=\cdots=\alpha_k=0$.

Since $A$ is a $\mathbb{Q}$-independent subset, by proposition 5.3 there exists a basis $B$ of $\mathbb{R}$ that contains $A$. Then $A\subseteq B\subseteq\mathbb{R}$ and $|A|=|\mathbb{R}|$ and Cantor-Bernstein theorem imply $|B|=|\mathbb{R}|$, therefore $[\mathbb{R}:\mathbb{Q}]=\mathrm{dim}_\mathbb{Q}{}\mathbb{R}=|\mathbb{R}|$. $\quad\blacksquare$

-
Is this uncountable linearly independent set basis ? – bhim sen Chaudhary Jun 5 '15 at 2:53

## protected by user26857Nov 25 '15 at 9:30

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site.

Would you like to answer one of these unanswered questions instead?


# Iterating distributions

I want to calculate the distribution that comes from summing up random numbers between -0.5 and 0.5.

A computationally intensive version of doing this is to simply do the following:

SmoothHistogram[
Table[Table[
Sum[RandomReal[] - 0.5, {i, 1,

This won't work however if the number of numbers to be summed is very large, like a 1000000 for example.

So I tried doing it by first calculating the distribution of one random number, then calculating the distribution of two random numbers from the first distribution. The first distribution is simply:

UnitBox[x]

The next distribution can be calculated by integrating the previous distribution from x-0.5 to x+0.5:

Integrate[UnitBox[a], {a, x - 1/2, x + 1/2}]

I can't figure out how to iterate this to, for example, draw a plot from the result.

I also tried the following:

NestList[PDF[
ProbabilityDistribution[
Integrate[# /. x -> a, {a, x - 1/2, x + 1/2},
Assumptions -> x \[Element] Reals], {x, -2, 2}], x] &,
UnitBox[x], 5]
Plot[%, {x, -2, 2}]

• Any reason not to use built-ins? – ciao Apr 19 '15 at 3:26
• No. What built-in would help? – user Apr 19 '15 at 3:27
• maybe UniformSumDistribution. E.g., dis[n_] := UniformSumDistribution[n, {-.5, .5}]; Plot[Evaluate[PDF[dis@#, x] & /@ Range[10]], {x, -2, 2}]? – kglr Apr 19 '15 at 3:30
• It works! Thanks a lot. – user Apr 19 '15 at 3:32
• @MathematicaUser39386, posted the comment as an answer. – kglr Apr 19 '15 at 4:03

• summing random numbers between -0.5 and 0.5.

• the number of numbers to be summed is very large, like a 1000000 for example.

The sum of $n$ identical Uniform random variables is known as a generalised Irwin-Hall distribution, implemented in Mathematica as the UniformSumDistribution [ see kguler's answer]. The latter takes an $n$-part piecewise form, so, for example, if $n = 10$, the pdf is:

PDF[UniformSumDistribution[10, {-1/2, 1/2}], x]

The above is exact and works beautifully for small $n$. However, the OP poses the problem of $n$ being very large -- such as $n$ = 1 million -- which involves creating a 1 million part piecewise structure which is certainly going to fail. In fact, for $n = 1000$, simply making a plot of the pdf takes about 400 seconds on my Mac Pro:

(AA = Plot[
PDF[UniformSumDistribution[1000, {-1/2, 1/2}], x], {x, -50, 50},
PlotRange -> All]) // AbsoluteTiming

390 seconds

• For $n=10,000$, the above plot is simply unworkable ... which leaves the OP's question: how to proceed for large $n$??

Large $n$: apply Central Limit Theorem

For large $n$, I would suggest applying the (Lindeberg-Lévy) Central Limit Theorem:

• If the random variables $(X_1, X_2, \dots)$ are independent and identically distributed, each with finite mean $\mu$ and finite variance $\sigma^2$, then the sample sum:

$$S_n \overset{a}{\sim} N\left(n \mu,n \sigma ^2\right)$$

In our case, $X \sim \text{Uniform}(-\frac12,\frac12)$ so $\mu = 0$ and $\sigma^2 = \frac{1}{12}$, so the asymptotic distribution of the sample sum is:

$$S_n \overset{a}{\sim} N\left(0, \frac{n}{12} \right)$$

with pdf $f(x)$:

       f = Exp[-6 (x^2/n)] / Sqrt[n Pi /6];
domain[f] = {x, -Infinity, Infinity} && {n > 0};

Here is a plot of pdf $f(x)$ when $n = 1000$:

BB = Plot[f /. n -> 1000, {x, -50, 50}, PlotRange -> All, PlotStyle -> Red]

This is, of course, instantaneous and works for arbitrarily large $n$. The following plot compares the exact solution AA to the asymptotic solution BB when $n = 1000$:

Show[BB, AA]

There is no discernible visual difference between the two plots here. Whereas the exact AA solution fails to evaluate for very large $n$, the asymptotic BB solution will always evaluate immediately, and with ever improving accuracy as $n$ increases.

The built-in function UniformSumDistribution may be useful:

usd[n_] := UniformSumDistribution[n, {-.5, .5}];
Plot[Evaluate[PDF[usd@#, x] & /@ Range[20]], {x, -2, 2},
PlotStyle -> (ColorData[{"Rainbow", {1, 20}}] /@ Range[20]),
Exclusions -> None, PlotRange -> All, ImageSize -> 500,
PlotLegends -> ("usd (" <> ToString[#] <> ")" & /@ Range[20])]

ClearAll[skd]
skd[n_] := SmoothKernelDistribution[RandomVariate[usd[n], 5000]];
Plot[Evaluate[PDF[#, x] & /@ {skd[5], usd[5], skd[3], usd[3]}], {x, -2, 2},
ImageSize -> 500, Exclusions -> None,
Filling -> {1 -> {{2}, {Orange, Yellow}}, 3 -> {{4}, {Blue, Green}}}]


We are working to support a site-wide PDF but it is not yet available. You can download PDFs for individual lectures through the download badge on each lecture page.

Code should execute sequentially if run in a Jupyter notebook

# Linear Algebra¶

## Overview¶

Linear algebra is one of the most useful branches of applied mathematics for economists to invest in

For example, many applied problems in economics and finance require the solution of a linear system of equations, such as

$$\begin{array}{c} y_1 = a x_1 + b x_2 \\ y_2 = c x_1 + d x_2 \end{array}$$

or, more generally,

$$\begin{array}{c} y_1 = a_{11} x_1 + a_{12} x_2 + \cdots + a_{1k} x_k \\ \vdots \\ y_n = a_{n1} x_1 + a_{n2} x_2 + \cdots + a_{nk} x_k \end{array} \tag{1}$$

The objective here is to solve for the “unknowns” $x_1, \ldots, x_k$ given $a_{11}, \ldots, a_{nk}$ and $y_1, \ldots, y_n$

When considering such problems, it is essential that we first consider at least some of the following questions

• Does a solution actually exist?
• Are there in fact many solutions, and if so how should we interpret them?
• If no solution exists, is there a best “approximate” solution?
• If a solution exists, how should we compute it?

These are the kinds of topics addressed by linear algebra

In this lecture we will cover the basics of linear and matrix algebra, treating both theory and computation

We admit some overlap with this lecture, where operations on NumPy arrays were first explained

Note that this lecture is more theoretical than most, and contains background material that will be used in applications as we go along

## Vectors¶

A vector of length $n$ is just a sequence (or array, or tuple) of $n$ numbers, which we write as $x = (x_1, \ldots, x_n)$ or $x = [x_1, \ldots, x_n]$

We will write these sequences either horizontally or vertically as we please

(Later, when we wish to perform certain matrix operations, it will become necessary to distinguish between the two)

The set of all $n$-vectors is denoted by $\mathbb R^n$

For example, $\mathbb R ^2$ is the plane, and a vector in $\mathbb R^2$ is just a point in the plane

Traditionally, vectors are represented visually as arrows from the origin to the point

The following figure represents three vectors in this manner

In [1]:
import matplotlib.pyplot as plt
%matplotlib inline

fig, ax = plt.subplots(figsize=(10, 8))
# Set the axes through the origin
for spine in ['left', 'bottom']:
ax.spines[spine].set_position('zero')
for spine in ['right', 'top']:
ax.spines[spine].set_color('none')

ax.set(xlim=(-5, 5), ylim=(-5, 5))
ax.grid()
vecs = ((2, 4), (-3, 3), (-4, -3.5))
for v in vecs:
ax.annotate('', xy=v, xytext=(0, 0),
arrowprops=dict(facecolor='blue',
shrink=0,
alpha=0.7,
width=0.5))
ax.text(1.1 * v[0], 1.1 * v[1], str(v))
plt.show()

### Vector Operations¶

The two most common operators for vectors are addition and scalar multiplication, which we now describe

As a matter of definition, when we add two vectors, we add them element by element

$$x + y = \left[ \begin{array}{c} x_1 \\ x_2 \\ \vdots \\ x_n \end{array} \right] + \left[ \begin{array}{c} y_1 \\ y_2 \\ \vdots \\ y_n \end{array} \right] := \left[ \begin{array}{c} x_1 + y_1 \\ x_2 + y_2 \\ \vdots \\ x_n + y_n \end{array} \right]$$

Scalar multiplication is an operation that takes a number $\gamma$ and a vector $x$ and produces

$$\gamma x := \left[ \begin{array}{c} \gamma x_1 \\ \gamma x_2 \\ \vdots \\ \gamma x_n \end{array} \right]$$

Scalar multiplication is illustrated in the next figure

In [2]:
import numpy as np

fig, ax = plt.subplots(figsize=(10, 8))
# Set the axes through the origin
for spine in ['left', 'bottom']:
ax.spines[spine].set_position('zero')
for spine in ['right', 'top']:
ax.spines[spine].set_color('none')

ax.set(xlim=(-5, 5), ylim=(-5, 5))
x = (2, 2)
ax.annotate('', xy=x, xytext=(0, 0),
arrowprops=dict(facecolor='blue',
shrink=0,
alpha=1,
width=0.5))
ax.text(x[0] + 0.4, x[1] - 0.2, '$x$', fontsize='16')

scalars = (-2, 2)
x = np.array(x)

for s in scalars:
v = s * x
ax.annotate('', xy=v, xytext=(0, 0),
arrowprops=dict(facecolor='red',
shrink=0,
alpha=0.5,
width=0.5))
ax.text(v[0] + 0.4, v[1] - 0.2, f'${s} x$', fontsize='16')
plt.show()

In Python, a vector can be represented as a list or tuple, such as x = (2, 4, 6), but is more commonly represented as a NumPy array

One advantage of NumPy arrays is that scalar multiplication and addition have very natural syntax

In [3]:
x = np.ones(3)            # Vector of three ones
y = np.array((2, 4, 6))   # Converts tuple (2, 4, 6) into array
x + y
Out[3]:
array([3., 5., 7.])
In [4]:
4 * x
Out[4]:
array([4., 4., 4.])

### Inner Product and Norm¶

The inner product of vectors $x,y \in \mathbb R ^n$ is defined as

$$x' y := \sum_{i=1}^n x_i y_i$$

Two vectors are called orthogonal if their inner product is zero

The norm of a vector $x$ represents its “length” (i.e., its distance from the zero vector) and is defined as

$$\| x \| := \sqrt{x' x} := \left( \sum_{i=1}^n x_i^2 \right)^{1/2}$$

The expression $\| x - y\|$ is thought of as the distance between $x$ and $y$

Continuing on from the previous example, the inner product and norm can be computed as follows

In [5]:
np.sum(x * y)          # Inner product of x and y
Out[5]:
12.0
In [6]:
np.sqrt(np.sum(x**2))  # Norm of x, take one
Out[6]:
1.7320508075688772
In [7]:
np.linalg.norm(x)      # Norm of x, take two
Out[7]:
1.7320508075688772

### Span¶

Given a set of vectors $A := \{a_1, \ldots, a_k\}$ in $\mathbb R ^n$, it’s natural to think about the new vectors we can create by performing linear operations

New vectors created in this manner are called linear combinations of $A$

In particular, $y \in \mathbb R ^n$ is a linear combination of $A := \{a_1, \ldots, a_k\}$ if

$$y = \beta_1 a_1 + \cdots + \beta_k a_k \text{ for some scalars } \beta_1, \ldots, \beta_k$$

In this context, the values $\beta_1, \ldots, \beta_k$ are called the coefficients of the linear combination

The set of linear combinations of $A$ is called the span of $A$

The next figure shows the span of $A = \{a_1, a_2\}$ in $\mathbb R ^3$

The span is a 2 dimensional plane passing through these two points and the origin

In [8]:
from matplotlib import cm
from mpl_toolkits.mplot3d import Axes3D
from scipy.interpolate import interp2d

fig = plt.figure(figsize=(10, 8))
ax = fig.gca(projection='3d')

x_min, x_max = -5, 5
y_min, y_max = -5, 5

α, β = 0.2, 0.1

ax.set(xlim=(x_min, x_max), ylim=(x_min, x_max), zlim=(x_min, x_max),
xticks=(0,), yticks=(0,), zticks=(0,))

gs = 3
z = np.linspace(x_min, x_max, gs)
x = np.zeros(gs)
y = np.zeros(gs)
ax.plot(x, y, z, 'k-', lw=2, alpha=0.5)
ax.plot(z, x, y, 'k-', lw=2, alpha=0.5)
ax.plot(y, z, x, 'k-', lw=2, alpha=0.5)

# Fixed linear function, to generate a plane
def f(x, y):
return α * x + β * y

# Vector locations, by coordinate
x_coords = np.array((3, 3))
y_coords = np.array((4, -4))
z = f(x_coords, y_coords)
for i in (0, 1):
ax.text(x_coords[i], y_coords[i], z[i], f'$a_{i+1}$', fontsize=14)

# Lines to vectors
for i in (0, 1):
x = (0, x_coords[i])
y = (0, y_coords[i])
z = (0, f(x_coords[i], y_coords[i]))
ax.plot(x, y, z, 'b-', lw=1.5, alpha=0.6)

# Draw the plane
grid_size = 20
xr2 = np.linspace(x_min, x_max, grid_size)
yr2 = np.linspace(y_min, y_max, grid_size)
x2, y2 = np.meshgrid(xr2, yr2)
z2 = f(x2, y2)
ax.plot_surface(x2, y2, z2, rstride=1, cstride=1, cmap=cm.jet,
linewidth=0, antialiased=True, alpha=0.2)
plt.show()

#### Examples¶

If $A$ contains only one vector $a_1 \in \mathbb R ^2$, then its span is just the scalar multiples of $a_1$, which is the unique line passing through both $a_1$ and the origin

If $A = \{e_1, e_2, e_3\}$ consists of the canonical basis vectors of $\mathbb R ^3$, that is

$$e_1 := \left[ \begin{array}{c} 1 \\ 0 \\ 0 \end{array} \right] , \quad e_2 := \left[ \begin{array}{c} 0 \\ 1 \\ 0 \end{array} \right] , \quad e_3 := \left[ \begin{array}{c} 0 \\ 0 \\ 1 \end{array} \right]$$

then the span of $A$ is all of $\mathbb R ^3$, because, for any $x = (x_1, x_2, x_3) \in \mathbb R ^3$, we can write

$$x = x_1 e_1 + x_2 e_2 + x_3 e_3$$

Now consider $A_0 = \{e_1, e_2, e_1 + e_2\}$

If $y = (y_1, y_2, y_3)$ is any linear combination of these vectors, then $y_3 = 0$ (check it)

Hence $A_0$ fails to span all of $\mathbb R ^3$

### Linear Independence¶

As we’ll see, it’s often desirable to find families of vectors with relatively large span, so that many vectors can be described by linear operators on a few vectors

The condition we need for a set of vectors to have a large span is what’s called linear independence

In particular, a collection of vectors $A := \{a_1, \ldots, a_k\}$ in $\mathbb R ^n$ is said to be

• linearly dependent if some strict subset of $A$ has the same span as $A$
• linearly independent if it is not linearly dependent

Put differently, a set of vectors is linearly independent if no vector is redundant to the span, and linearly dependent otherwise

To illustrate the idea, recall the figure that showed the span of vectors $\{a_1, a_2\}$ in $\mathbb R ^3$ as a plane through the origin

If we take a third vector $a_3$ and form the set $\{a_1, a_2, a_3\}$, this set will be

• linearly dependent if $a_3$ lies in the plane
• linearly independent otherwise

As another illustration of the concept, since $\mathbb R ^n$ can be spanned by $n$ vectors (see the discussion of canonical basis vectors above), any collection of $m > n$ vectors in $\mathbb R ^n$ must be linearly dependent

The following statements are equivalent to linear independence of $A := \{a_1, \ldots, a_k\} \subset \mathbb R ^n$

1. No vector in $A$ can be formed as a linear combination of the other elements
2. If $\beta_1 a_1 + \cdots \beta_k a_k = 0$ for scalars $\beta_1, \ldots, \beta_k$, then $\beta_1 = \cdots = \beta_k = 0$

(The zero in the first expression is the origin of $\mathbb R ^n$)

### Unique Representations¶

Another nice thing about sets of linearly independent vectors is that each element in the span has a unique representation as a linear combination of these vectors

In other words, if $A := \{a_1, \ldots, a_k\} \subset \mathbb R ^n$ is linearly independent and

$$y = \beta_1 a_1 + \cdots \beta_k a_k$$

then no other coefficient sequence $\gamma_1, \ldots, \gamma_k$ will produce the same vector $y$

Indeed, if we also have $y = \gamma_1 a_1 + \cdots \gamma_k a_k$, then

$$(\beta_1 - \gamma_1) a_1 + \cdots + (\beta_k - \gamma_k) a_k = 0$$

Linear independence now implies $\gamma_i = \beta_i$ for all $i$

## Matrices¶

Matrices are a neat way of organizing data for use in linear operations

An $n \times k$ matrix is a rectangular array $A$ of numbers with $n$ rows and $k$ columns:

$$A = \left[ \begin{array}{cccc} a_{11} & a_{12} & \cdots & a_{1k} \\ a_{21} & a_{22} & \cdots & a_{2k} \\ \vdots & \vdots & & \vdots \\ a_{n1} & a_{n2} & \cdots & a_{nk} \end{array} \right]$$

Often, the numbers in the matrix represent coefficients in a system of linear equations, as discussed at the start of this lecture

For obvious reasons, the matrix $A$ is also called a vector if either $n = 1$ or $k = 1$

In the former case, $A$ is called a row vector, while in the latter it is called a column vector

If $n = k$, then $A$ is called square

The matrix formed by replacing $a_{ij}$ by $a_{ji}$ for every $i$ and $j$ is called the transpose of $A$, and denoted $A'$ or $A^{\top}$

If $A = A'$, then $A$ is called symmetric

For a square matrix $A$, the $i$ elements of the form $a_{ii}$ for $i=1,\ldots,n$ are called the principal diagonal

$A$ is called diagonal if the only nonzero entries are on the principal diagonal

If, in addition to being diagonal, each element along the principal diagonal is equal to 1, then $A$ is called the identity matrix, and denoted by $I$

### Matrix Operations¶

Just as was the case for vectors, a number of algebraic operations are defined for matrices

Scalar multiplication and addition are immediate generalizations of the vector case:

$$\gamma A = \gamma \left[ \begin{array}{ccc} a_{11} & \cdots & a_{1k} \\ \vdots & \vdots & \vdots \\ a_{n1} & \cdots & a_{nk} \\ \end{array} \right] := \left[ \begin{array}{ccc} \gamma a_{11} & \cdots & \gamma a_{1k} \\ \vdots & \vdots & \vdots \\ \gamma a_{n1} & \cdots & \gamma a_{nk} \\ \end{array} \right]$$

and

$$A + B = \left[ \begin{array}{ccc} a_{11} & \cdots & a_{1k} \\ \vdots & \vdots & \vdots \\ a_{n1} & \cdots & a_{nk} \\ \end{array} \right] + \left[ \begin{array}{ccc} b_{11} & \cdots & b_{1k} \\ \vdots & \vdots & \vdots \\ b_{n1} & \cdots & b_{nk} \\ \end{array} \right] := \left[ \begin{array}{ccc} a_{11} + b_{11} & \cdots & a_{1k} + b_{1k} \\ \vdots & \vdots & \vdots \\ a_{n1} + b_{n1} & \cdots & a_{nk} + b_{nk} \\ \end{array} \right]$$

In the latter case, the matrices must have the same shape in order for the definition to make sense

We also have a convention for multiplying two matrices

The rule for matrix multiplication generalizes the idea of inner products discussed above, and is designed to make multiplication play well with basic linear operations

If $A$ and $B$ are two matrices, then their product $A B$ is formed by taking as its $i,j$-th element the inner product of the $i$-th row of $A$ and the $j$-th column of $B$

If $A$ is $n \times k$ and $B$ is $j \times m$, then to multiply $A$ and $B$ we require $k = j$, and the resulting matrix $A B$ is $n \times m$

As perhaps the most important special case, consider multiplying $n \times k$ matrix $A$ and $k \times 1$ column vector $x$

According to the preceding rule, this gives us an $n \times 1$ column vector

$$A x = \left[ \begin{array}{ccc} a_{11} & \cdots & a_{1k} \\ \vdots & \vdots & \vdots \\ a_{n1} & \cdots & a_{nk} \end{array} \right] \left[ \begin{array}{c} x_{1} \\ \vdots \\ x_{k} \end{array} \right] := \left[ \begin{array}{c} a_{11} x_1 + \cdots + a_{1k} x_k \\ \vdots \\ a_{n1} x_1 + \cdots + a_{nk} x_k \end{array} \right] \tag{2}$$

Note

$A B$ and $B A$ are not generally the same thing

Another important special case is the identity matrix

You should check that if $A$ is $n \times k$ and $I$ is the $k \times k$ identity matrix, then $AI = A$

If $I$ is the $n \times n$ identity matrix, then $IA = A$

### Matrices in NumPy¶

NumPy arrays are also used as matrices, and have fast, efficient functions and methods for all the standard matrix operations [1]

You can create them manually from tuples of tuples (or lists of lists) as follows

In [9]:
A = ((1, 2),
(3, 4))

type(A)
Out[9]:
tuple
In [10]:
A = np.array(A)

type(A)
Out[10]:
numpy.ndarray
In [11]:
A.shape
Out[11]:
(2, 2)

The shape attribute is a tuple giving the number of rows and columns — see here for more discussion

To get the transpose of A, use A.transpose() or, more simply, A.T

There are many convenient functions for creating common matrices (matrices of zeros, ones, etc.) — see here

Since operations are performed elementwise by default, scalar multiplication and addition have very natural syntax

In [12]:
A = np.identity(3)
B = np.ones((3, 3))
2 * A
Out[12]:
array([[2., 0., 0.],
[0., 2., 0.],
[0., 0., 2.]])
In [13]:
A + B
Out[13]:
array([[2., 1., 1.],
[1., 2., 1.],
[1., 1., 2.]])

To multiply matrices we use the @ symbol

In particular, A @ B is matrix multiplication, whereas A * B is element by element multiplication

See here for more discussion

### Matrices as Maps¶

Each $n \times k$ matrix $A$ can be identified with a function $f(x) = Ax$ that maps $x \in \mathbb R ^k$ into $y = Ax \in \mathbb R ^n$

These kinds of functions have a special property: they are linear

A function $f \colon \mathbb R ^k \to \mathbb R ^n$ is called linear if, for all $x, y \in \mathbb R ^k$ and all scalars $\alpha, \beta$, we have

$$f(\alpha x + \beta y) = \alpha f(x) + \beta f(y)$$

You can check that this holds for the function $f(x) = A x + b$ when $b$ is the zero vector, and fails when $b$ is nonzero

In fact, it’s known that $f$ is linear if and only if there exists a matrix $A$ such that $f(x) = Ax$ for all $x$

## Solving Systems of Equations¶

Recall again the system of equations (1)

If we compare (1) and (2), we see that (1) can now be written more conveniently as

$$y = Ax \tag{3}$$

The problem we face is to determine a vector $x \in \mathbb R ^k$ that solves (3), taking $y$ and $A$ as given

This is a special case of a more general problem: Find an $x$ such that $y = f(x)$

Given an arbitrary function $f$ and a $y$, is there always an $x$ such that $y = f(x)$?

If so, is it always unique?

The answer to both these questions is negative, as the next figure shows

In [14]:
def f(x):
return 0.6 * np.cos(4 * x) + 1.4

xmin, xmax = -1, 1
x = np.linspace(xmin, xmax, 160)
y = f(x)
ya, yb = np.min(y), np.max(y)

fig, axes = plt.subplots(2, 1, figsize=(10, 10))

for ax in axes:
# Set the axes through the origin
for spine in ['left', 'bottom']:
ax.spines[spine].set_position('zero')
for spine in ['right', 'top']:
ax.spines[spine].set_color('none')

ax.set(ylim=(-0.6, 3.2), xlim=(xmin, xmax),
yticks=(), xticks=())

ax.plot(x, y, 'k-', lw=2, label='$f$')
ax.fill_between(x, ya, yb, facecolor='blue', alpha=0.05)
ax.vlines([0], ya, yb, lw=3, color='blue', label='range of $f$')
ax.text(0.04, -0.3, '$0$', fontsize=16)

ax = axes[0]

ax.legend(loc='upper right', frameon=False)
ybar = 1.5
ax.plot(x, x * 0 + ybar, 'k--', alpha=0.5)
ax.text(0.05, 0.8 * ybar, '$y$', fontsize=16)
for i, z in enumerate((-0.35, 0.35)):
ax.vlines(z, 0, f(z), linestyle='--', alpha=0.5)
ax.text(z, -0.2, f'$x_{i}$', fontsize=16)

ax = axes[1]

ybar = 2.6
ax.plot(x, x * 0 + ybar, 'k--', alpha=0.5)
ax.text(0.04, 0.91 * ybar, '$y$', fontsize=16)

plt.show()

In the first plot there are multiple solutions, as the function is not one-to-one, while in the second there are no solutions, since $y$ lies outside the range of $f$

Can we impose conditions on $A$ in (3) that rule out these problems?

In this context, the most important thing to recognize about the expression $Ax$ is that it corresponds to a linear combination of the columns of $A$

In particular, if $a_1, \ldots, a_k$ are the columns of $A$, then

$$Ax = x_1 a_1 + \cdots + x_k a_k$$

Hence the range of $f(x) = Ax$ is exactly the span of the columns of $A$

We want the range to be large, so that it contains arbitrary $y$

As you might recall, the condition that we want for the span to be large is linear independence

A happy fact is that linear independence of the columns of $A$ also gives us uniqueness

Indeed, it follows from our earlier discussion that if $\{a_1, \ldots, a_k\}$ are linearly independent and $y = Ax = x_1 a_1 + \cdots + x_k a_k$, then no $z \not= x$ satisfies $y = Az$

### The $n \times n$ Case¶

Let’s discuss some more details, starting with the case where $A$ is $n \times n$

This is the familiar case where the number of unknowns equals the number of equations

For arbitrary $y \in \mathbb R ^n$, we hope to find a unique $x \in \mathbb R ^n$ such that $y = Ax$

In view of the observations immediately above, if the columns of $A$ are linearly independent, then their span, and hence the range of $f(x) = Ax$, is all of $\mathbb R ^n$

Hence there always exists an $x$ such that $y = Ax$

Moreover, the solution is unique

In particular, the following are equivalent

1. The columns of $A$ are linearly independent
2. For any $y \in \mathbb R ^n$, the equation $y = Ax$ has a unique solution

The property of having linearly independent columns is sometimes expressed as having full column rank

#### Inverse Matrices¶

Can we give some sort of expression for the solution?

If $y$ and $A$ are scalar with $A \not= 0$, then the solution is $x = A^{-1} y$

A similar expression is available in the matrix case

In particular, if square matrix $A$ has full column rank, then it possesses a multiplicative inverse matrix $A^{-1}$, with the property that $A A^{-1} = A^{-1} A = I$

As a consequence, if we pre-multiply both sides of $y = Ax$ by $A^{-1}$, we get $x = A^{-1} y$

This is the solution that we’re looking for

#### Determinants¶

Another quick comment about square matrices is that to every such matrix we assign a unique number called the determinant of the matrix — you can find the expression for it here

If the determinant of $A$ is not zero, then we say that $A$ is nonsingular

Perhaps the most important fact about determinants is that $A$ is nonsingular if and only if $A$ is of full column rank

This gives us a useful one-number summary of whether or not a square matrix can be inverted

### More Rows than Columns¶

This is the $n \times k$ case with $n > k$

This case is very important in many settings, not least in the setting of linear regression (where $n$ is the number of observations, and $k$ is the number of explanatory variables)

Given arbitrary $y \in \mathbb R ^n$, we seek an $x \in \mathbb R ^k$ such that $y = Ax$

In this setting, existence of a solution is highly unlikely

Without much loss of generality, let’s go over the intuition focusing on the case where the columns of $A$ are linearly independent

It follows that the span of the columns of $A$ is a $k$-dimensional subspace of $\mathbb R ^n$

This span is very “unlikely” to contain arbitrary $y \in \mathbb R ^n$

To see why, recall the figure above, where $k=2$ and $n=3$

Imagine an arbitrarily chosen $y \in \mathbb R ^3$, located somewhere in that three dimensional space

What’s the likelihood that $y$ lies in the span of $\{a_1, a_2\}$ (i.e., the two dimensional plane through these points)?

In a sense it must be very small, since this plane has zero “thickness”

As a result, in the $n > k$ case we usually give up on existence

However, we can still seek a best approximation, for example an $x$ that makes the distance $\| y - Ax\|$ as small as possible

To solve this problem, one can use either calculus or the theory of orthogonal projections

The solution is known to be $\hat x = (A'A)^{-1}A'y$ — see for example chapter 3 of these notes

### More Columns than Rows¶

This is the $n \times k$ case with $n < k$, so there are fewer equations than unknowns

In this case there are either no solutions or infinitely many — in other words, uniqueness never holds

For example, consider the case where $k=3$ and $n=2$

Thus, the columns of $A$ consists of 3 vectors in $\mathbb R ^2$

This set can never be linearly independent, since it is possible to find two vectors that span $\mathbb R ^2$

(For example, use the canonical basis vectors)

It follows that one column is a linear combination of the other two

For example, let’s say that $a_1 = \alpha a_2 + \beta a_3$

Then if $y = Ax = x_1 a_1 + x_2 a_2 + x_3 a_3$, we can also write

$$y = x_1 (\alpha a_2 + \beta a_3) + x_2 a_2 + x_3 a_3 = (x_1 \alpha + x_2) a_2 + (x_1 \beta + x_3) a_3$$

In other words, uniqueness fails

### Linear Equations with SciPy¶

Here’s an illustration of how to solve linear equations with SciPy’s linalg submodule

All of these routines are Python front ends to time-tested and highly optimized FORTRAN code

In [15]:
from scipy.linalg import inv, solve, det

A = ((1, 2), (3, 4))
A = np.array(A)
y = np.ones((2, 1))  # Column vector
det(A)  # Check that A is nonsingular, and hence invertible
Out[15]:
-2.0
In [16]:
A_inv = inv(A)  # Compute the inverse
A_inv
Out[16]:
array([[-2. ,  1. ],
[ 1.5, -0.5]])
In [17]:
x = A_inv @ y  # Solution
A @ x          # Should equal y
Out[17]:
array([[1.],
[1.]])
In [18]:
solve(A, y)  # Produces same solution
Out[18]:
array([[-1.],
[ 1.]])

Observe how we can solve for $x = A^{-1} y$ by either via inv(A) @ y, or using solve(A, y)

The latter method uses a different algorithm (LU decomposition) that is numerically more stable, and hence should almost always be preferred

To obtain the least squares solution $\hat x = (A'A)^{-1}A'y$, use scipy.linalg.lstsq(A, y)

## Eigenvalues and Eigenvectors¶

Let $A$ be an $n \times n$ square matrix

If $\lambda$ is scalar and $v$ is a non-zero vector in $\mathbb R ^n$ such that

$$A v = \lambda v$$

then we say that $\lambda$ is an eigenvalue of $A$, and $v$ is an eigenvector

Thus, an eigenvector of $A$ is a vector such that when the map $f(x) = Ax$ is applied, $v$ is merely scaled

The next figure shows two eigenvectors (blue arrows) and their images under $A$ (red arrows)

As expected, the image $Av$ of each $v$ is just a scaled version of the original

In [19]:
from scipy.linalg import eig

A = ((1, 2),
(2, 1))
A = np.array(A)
evals, evecs = eig(A)
evecs = evecs[:, 0], evecs[:, 1]

fig, ax = plt.subplots(figsize=(10, 8))
# Set the axes through the origin
for spine in ['left', 'bottom']:
ax.spines[spine].set_position('zero')
for spine in ['right', 'top']:
ax.spines[spine].set_color('none')
ax.grid(alpha=0.4)

xmin, xmax = -3, 3
ymin, ymax = -3, 3
ax.set(xlim=(xmin, xmax), ylim=(ymin, ymax))

# Plot each eigenvector
for v in evecs:
ax.annotate('', xy=v, xytext=(0, 0),
arrowprops=dict(facecolor='blue',
shrink=0,
alpha=0.6,
width=0.5))

# Plot the image of each eigenvector
for v in evecs:
v = A @ v
ax.annotate('', xy=v, xytext=(0, 0),
arrowprops=dict(facecolor='red',
shrink=0,
alpha=0.6,
width=0.5))

# Plot the lines they run through
x = np.linspace(xmin, xmax, 3)
for v in evecs:
a = v[1] / v[0]
ax.plot(x, a * x, 'b-', lw=0.4)

plt.show()

The eigenvalue equation is equivalent to $(A - \lambda I) v = 0$, and this has a nonzero solution $v$ only when the columns of $A - \lambda I$ are linearly dependent

This in turn is equivalent to stating that the determinant is zero

Hence to find all eigenvalues, we can look for $\lambda$ such that the determinant of $A - \lambda I$ is zero

This problem can be expressed as one of solving for the roots of a polynomial in $\lambda$ of degree $n$

This in turn implies the existence of $n$ solutions in the complex plane, although some might be repeated

Some nice facts about the eigenvalues of a square matrix $A$ are as follows

1. The determinant of $A$ equals the product of the eigenvalues
2. The trace of $A$ (the sum of the elements on the principal diagonal) equals the sum of the eigenvalues
3. If $A$ is symmetric, then all of its eigenvalues are real
4. If $A$ is invertible and $\lambda_1, \ldots, \lambda_n$ are its eigenvalues, then the eigenvalues of $A^{-1}$ are $1/\lambda_1, \ldots, 1/\lambda_n$

A corollary of the first statement is that a matrix is invertible if and only if all its eigenvalues are nonzero

Using SciPy, we can solve for the eigenvalues and eigenvectors of a matrix as follows

In [20]:
A = ((1, 2),
(2, 1))

A = np.array(A)
evals, evecs = eig(A)
evals
Out[20]:
array([ 3.+0.j, -1.+0.j])
In [21]:
evecs
Out[21]:
array([[ 0.70710678, -0.70710678],
[ 0.70710678,  0.70710678]])

Note that the columns of evecs are the eigenvectors

Since any scalar multiple of an eigenvector is an eigenvector with the same eigenvalue (check it), the eig routine normalizes the length of each eigenvector to one

### Generalized Eigenvalues¶

It is sometimes useful to consider the generalized eigenvalue problem, which, for given matrices $A$ and $B$, seeks generalized eigenvalues $\lambda$ and eigenvectors $v$ such that

$$A v = \lambda B v$$

This can be solved in SciPy via scipy.linalg.eig(A, B)

Of course, if $B$ is square and invertible, then we can treat the generalized eigenvalue problem as an ordinary eigenvalue problem $B^{-1} A v = \lambda v$, but this is not always the case

## Further Topics¶

We round out our discussion by briefly mentioning several other important topics

### Series Expansions¶

Recall the usual summation formula for a geometric progression, which states that if $|a| < 1$, then $\sum_{k=0}^{\infty} a^k = (1 - a)^{-1}$

A generalization of this idea exists in the matrix setting

#### Matrix Norms¶

Let $A$ be a square matrix, and let

$$\| A \| := \max_{\| x \| = 1} \| A x \|$$

The norms on the right-hand side are ordinary vector norms, while the norm on the left-hand side is a matrix norm — in this case, the so-called spectral norm

For example, for a square matrix $S$, the condition $\| S \| < 1$ means that $S$ is contractive, in the sense that it pulls all vectors towards the origin [2]

#### Neumann’s Theorem¶

Let $A$ be a square matrix and let $A^k := A A^{k-1}$ with $A^1 := A$

In other words, $A^k$ is the $k$-th power of $A$

Neumann’s theorem states the following: If $\| A^k \| < 1$ for some $k \in \mathbb{N}$, then $I - A$ is invertible, and

$$(I - A)^{-1} = \sum_{k=0}^{\infty} A^k \tag{4}$$

A result known as Gelfand’s formula tells us that, for any square matrix $A$,

$$\rho(A) = \lim_{k \to \infty} \| A^k \|^{1/k}$$

Here $\rho(A)$ is the spectral radius, defined as $\max_i |\lambda_i|$, where $\{\lambda_i\}_i$ is the set of eigenvalues of $A$

As a consequence of Gelfand’s formula, if all eigenvalues are strictly less than one in modulus, there exists a $k$ with $\| A^k \| < 1$

In which case (4) is valid

### Positive Definite Matrices¶

Let $A$ be a symmetric $n \times n$ matrix

We say that $A$ is

1. positive definite if $x' A x > 0$ for every $x \in \mathbb R ^n \setminus \{0\}$
2. positive semi-definite or nonnegative definite if $x' A x \geq 0$ for every $x \in \mathbb R ^n$

Analogous definitions exist for negative definite and negative semi-definite matrices

It is notable that if $A$ is positive definite, then all of its eigenvalues are strictly positive, and hence $A$ is invertible (with positive definite inverse)

### Differentiating Linear and Quadratic forms¶

The following formulas are useful in many economic contexts. Let

• $z, x$ and $a$ all be $n \times 1$ vectors
• $A$ be an $n \times n$ matrix
• $B$ be an $m \times n$ matrix and $y$ be an $m \times 1$ vector

Then

1. $\frac{\partial a' x}{\partial x} = a$
2. $\frac{\partial A x}{\partial x} = A'$
3. $\frac{\partial x'A x}{\partial x} = (A + A') x$
4. $\frac{\partial y'B z}{\partial y} = B z$
5. $\frac{\partial y'B z}{\partial B} = y z'$

Exercise 1 below asks you to apply these formulas

The documentation of the scipy.linalg submodule can be found here

Chapters 2 and 3 of the Econometric Theory contains a discussion of linear algebra along the same lines as above, with solved exercises

If you don’t mind a slightly abstract approach, a nice intermediate-level text on linear algebra is [Janich94]

## Exercises¶

### Exercise 1¶

Let $x$ be a given $n \times 1$ vector and consider the problem

$$v(x) = \max_{y,u} \left\{ - y'P y - u' Q u \right\}$$

subject to the linear constraint

$$y = A x + B u$$

Here

• $P$ is an $n \times n$ matrix and $Q$ is an $m \times m$ matrix
• $A$ is an $n \times n$ matrix and $B$ is an $n \times m$ matrix
• both $P$ and $Q$ are symmetric and positive semidefinite

(What must the dimensions of $y$ and $u$ be to make this a well-posed problem?)

One way to solve the problem is to form the Lagrangian

$$\mathcal L = - y' P y - u' Q u + \lambda' \left[A x + B u - y\right]$$

where $\lambda$ is an $n \times 1$ vector of Lagrange multipliers

Try applying the formulas given above for differentiating quadratic and linear forms to obtain the first-order conditions for maximizing $\mathcal L$ with respect to $y, u$ and minimizing it with respect to $\lambda$

Show that these conditions imply that

1. $\lambda = - 2 P y$
2. The optimizing choice of $u$ satisfies $u = - (Q + B' P B)^{-1} B' P A x$
3. The function $v$ satisfies $v(x) = - x' \tilde P x$ where $\tilde P = A' P A - A'P B (Q + B'P B)^{-1} B' P A$

As we will see, in economic contexts Lagrange multipliers often are shadow prices

Note

If we don’t care about the Lagrange multipliers, we can substitute the constraint into the objective function, and then just maximize $-(Ax + Bu)'P (Ax + Bu) - u' Q u$ with respect to $u$. You can verify that this leads to the same maximizer.

## Solutions¶

### Solution to Exercise 1¶

We have an optimization problem:

$$v(x) = \max_{y,u} \{ -y'Py - u'Qu \}$$

s.t.

$$y = Ax + Bu$$

with primitives

• $P$ be a symmetric and positive semidefinite $n \times n$ matrix
• $Q$ be a symmetric and positive semidefinite $m \times m$ matrix
• $A$ an $n \times n$ matrix
• $B$ an $n \times m$ matrix

The associated Lagrangian is :

$$L = -y'Py - u'Qu + \lambda' \lbrack Ax + Bu - y \rbrack$$

#### 1.¶

Differentiating Lagrangian equation w.r.t y and setting its derivative equal to zero yields

$$\frac{ \partial L}{\partial y} = - (P + P') y - \lambda = - 2 P y - \lambda = 0 \:,$$

since P is symmetric

Accordingly, the first-order condition for maximizing L w.r.t. y implies

$$\lambda = -2 Py \:$$

#### 2.¶

Differentiating Lagrangian equation w.r.t. u and setting its derivative equal to zero yields

$$\frac{ \partial L}{\partial u} = - (Q + Q') u - B'\lambda = - 2Qu + B'\lambda = 0 \:$$

Substituting $\lambda = -2 P y$ gives

$$Qu + B'Py = 0 \:$$

Substituting the linear constraint $y = Ax + Bu$ into above equation gives

$$Qu + B'P(Ax + Bu) = 0$$$$(Q + B'PB)u + B'PAx = 0$$

which is the first-order condition for maximizing L w.r.t. u

Thus, the optimal choice of u must satisfy

$$u = -(Q + B'PB)^{-1}B'PAx \:,$$

which follows from the definition of the first-order conditions for Lagrangian equation

#### 3.¶

Rewriting our problem by substituting the constraint into the objective function, we get

$$v(x) = \max_{u} \{ -(Ax+ Bu)'P(Ax+Bu) - u'Qu \} \:$$

Since we know the optimal choice of u satisfies $u = -(Q + B’PB)^{-1}B’PAx$, then

$$v(x) = -(Ax+ B u)'P(Ax+B u) - u'Q u \,\,\,\, with \,\,\,\, u = -(Q + B'PB)^{-1}B'PAx$$

To evaluate the function

\begin{aligned} v(x) &= -(Ax+ B u)'P(Ax+Bu) - u'Q u \\ &= -(x'A' + u'B')P(Ax+Bu) - u'Q u \\ &= - x'A'PAx - u'B'PAx - x'A'PBu - u'B'PBu - u'Qu \\ &= - x'A'PAx - 2u'B'PAx - u'(Q + B'PB) u \end{aligned}

For simplicity, denote by $S := (Q + B'PB)^{-1} B'PA$, then $u = -Sx$

Regarding the second term $- 2u'B'PAx$,

\begin{aligned} -2u'B'PAx &= -2 x'S'B'PAx \\ & = 2 x'A'PB( Q + B'PB)^{-1} B'PAx \end{aligned}

Notice that the term $(Q + B'PB)^{-1}$ is symmetric as both P and Q are symmetric

Regarding the third term $- u'(Q + B'PB) u$,

\begin{aligned} -u'(Q + B'PB) u &= - x'S' (Q + B'PB)Sx \\ &= -x'A'PB(Q + B'PB)^{-1}B'PAx \end{aligned}

Hence, the summation of second and third terms is $x'A'PB(Q + B'PB)^{-1}B'PAx$

This implies that

\begin{aligned} v(x) &= - x'A'PAx - 2u'B'PAx - u'(Q + B'PB) u\\ &= - x'A'PAx + x'A'PB(Q + B'PB)^{-1}B'PAx \\ &= -x'[A'PA - A'PB(Q + B'PB)^{-1}B'PA] x \end{aligned}

Therefore, the solution to the optimization problem $v(x) = -x' \tilde{P}x$ follows the above result by denoting $\tilde{P} := A'PA - A'PB(Q + B'PB)^{-1}B'PA$

Footnotes

[1] Although there is a specialized matrix data type defined in NumPy, it’s more standard to work with ordinary NumPy arrays. See this discussion.

[2] Suppose that $\|S \| < 1$. Take any nonzero vector $x$, and let $r := \|x\|$. We have $\| Sx \| = r \| S (x/r) \| \leq r \| S \| < r = \| x\|$. Hence every point is pulled towards the origin.

• Share page