stdevp( ) appears to be mislabeled
05-27-2016, 11:50 PM
Post: #1
 Mike Elzinga Junior Member Posts: 16 Joined: May 2016
stdevp( ) appears to be mislabeled
It appears that the stdevp( ) function is mislabeled.

Instead of being the population standard deviation, it is the sample standard deviation. The sample standard deviation has to be greater than the population standard deviaion by sqrt(N/(N-1)).

Here is a check: approx(sddev({1,2,3})) = 0.816496580928

stdevp({1,2,3}) = 1

stdevp({1,2,3})*sqrt(2/3) = 0.816496580928, which is the population standard deviation.

The quickest fix might be to relabel stdevp( ) as stdevs( ) and change help to reflect that this is the sample standard deviation rather than the population standard deviation.
05-28-2016, 01:00 AM
Post: #2
 Mike Elzinga Junior Member Posts: 16 Joined: May 2016
RE: stdevp( ) appears to be mislabeled
I made some typos in my post (vision issues; sorry).

The relevant functions are stddev( ) and stddevp( ).
05-28-2016, 02:11 AM
Post: #3
 Helge Gabert Senior Member Posts: 460 Joined: Dec 2013
RE: stdevp( ) appears to be mislabeled
That is the case.

The Stats 1 Var Numeric View "Stats" displays the sample and population standard deviation correctly; however, calls to stddev and stddevp are mixed up.
05-28-2016, 04:44 AM
Post: #4
 Tim Wessman Senior Member Posts: 2,148 Joined: Dec 2013
RE: stdevp( ) appears to be mislabeled
If I remember correctly, Bernard has it that way on purpose and does not think it should change. I'm not sure the reasoning myself.

TW

Although I work for the HP calculator group, the views and opinions I post here are my own.
05-28-2016, 07:11 AM
Post: #5
 parisse Senior Member Posts: 1,027 Joined: Dec 2013
RE: stdevp( ) appears to be mislabeled
stddevp = standard deviation of the population based on a sample
stddev = standard deviation
If this is too confusing, I have nothing against renaming stddevp to stddevs.
05-28-2016, 07:12 AM (This post was last modified: 05-28-2016 07:14 AM by salvomic.)
Post: #6
 salvomic Senior Member Posts: 1,365 Joined: Jan 2015
RE: stdevp( ) appears to be mislabeled
(05-28-2016 04:44 AM)Tim Wessman Wrote:  If I remember correctly, Bernard has it that way on purpose and does not think it should change. I'm not sure the reasoning myself.

IMHO stddev() should represent sX variable of Statistics 1Var and stddevp() the correspondent σX, otherwise another proposal is to rename stddevp() stddevs() ("of the sample) and in any case to adapt them to sX and σX.
However I consider also the Bernard's reasoning as stddev() and stdevp() are applied also to a matrix and so on...

Salvo

∫aL√0mic (IT9CLU), HP Prime 50g 41CX 71b 42s 12C 15C - DM42 WP34s :: Prime Soft. Lib
05-28-2016, 01:57 PM
Post: #7
 Mike Elzinga Junior Member Posts: 16 Joined: May 2016
RE: stdevp( ) appears to be mislabeled
The current notational standards in statistics are that σX is the population standard deviation and sX is the sample standard deviation. The Stats 1Var app is consistent with the notational standards; and it also agrees with the STATS app on the HP 50G, the TI 89 and TI 83 and 84, and various other statistics packages.

The reason for dividing by N-1 in the sample standard deviation is because we don't have either the actual population mean or the actual population standard deviation when we take a sample. By taking a sample, we hope we have values that are representative of the population so that we can use the sample mean and standard deviation as <i>estimates</i> of the corresponding population parameters. These estimates of population parameters that we take from our sample are called statistics.

However, we lose a degree of freedom in calculating the mean of the sample because we then turn right around and use that sample mean in calculating the sample standard deviation. This makes the sample standard deviation a biased estimate of the population standard deviation unless we divide by N-1 instead of N.

When you calculate the population mean and standard deviation, you have every indivitual in the population, so there is no conflict in subtracting individual measurements from the population mean. So in calculating the population standard deviation it is proper to divide by N.

However, if you apply the calculation to get the sample standard deviation to the population itself, you get a value that is sqrt(N/(N-1)) times larger than it is supposed to be. Remember that the sample is not the population; it is hopefully a representative sample of the population from which we hope to estimate the population parameters when we can't count every member of the population.

Also, when we look at the behaviors of the sample means and sample standard deviations as a function of sample size, we find that the standard deviations of the sample means decreases with sample size as σ/sqrt(N) whereas the standard deviation of the sample standard deviations decreases as σ/sqrt(2N) when sampling a population that has a normal distribution.

I discovered the inconsistency in the notion of stddevp when I was writing an app to demonstrate the behaviors of the sample mean and sample standard deviation as a function of sample size. :-)

So, to be consistent with standard statistical notation, I would think that stddevp should be the same as σX and stddev (or, better, stddevs) should be the same as sX. At the moment, they are reversed.
05-28-2016, 02:04 PM
Post: #8
 Mike Elzinga Junior Member Posts: 16 Joined: May 2016
RE: stdevp( ) appears to be mislabeled
I'm new to this site.

How does one get Greek and other mathematical symbols? HTML doesn't appear to work in my last post.
05-28-2016, 02:45 PM
Post: #9
 parisse Senior Member Posts: 1,027 Joined: Dec 2013
RE: stdevp( ) appears to be mislabeled
(05-28-2016 01:57 PM)Mike Elzinga Wrote:  So, to be consistent with standard statistical notation, I would think that stddevp should be the same as σX and stddev (or, better, stddevs) should be the same as sX. At the moment, they are reversed.
I don't see why stddev should correspond to sigmaX and stddevp to sX and not conversely. I find more natural to have the shortest name for the simplest definition and the longest name for the modification required to have an unbiaised estimation. That's why I want to keep stddev for division by sqrt(N). After thinking a bit about the name, stddevs might also be confusing, perhaps stddevpfroms would be better.
05-28-2016, 03:15 PM
Post: #10
 Mike Elzinga Junior Member Posts: 16 Joined: May 2016
RE: stdevp( ) appears to be mislabeled
(05-28-2016 02:45 PM)parisse Wrote:
(05-28-2016 01:57 PM)Mike Elzinga Wrote:  So, to be consistent with standard statistical notation, I would think that stddevp should be the same as σX and stddev (or, better, stddevs) should be the same as sX. At the moment, they are reversed.
I don't see why stddev should correspond to sigmaX and stddevp to sX and not conversely. I find more natural to have the shortest name for the simplest definition and the longest name for the modification required to have an unbiaised estimation. That's why I want to keep stddev for division by sqrt(N). After thinking a bit about the name, stddevs might also be confusing, perhaps stddevpfroms would be better.

Because this is also a pedagogical issue, the names should reflect what calculation is being done.

On other statistics packages and on the HP50G, population standard deviation has division by N and sample standard deviation has division by N-1.

Standard notation in statistics these days has sigma for the population standard deviation and s for the sample standard deviation.

The help given for stddevp on the HP Prime says it is the population standard deviation. However, it actually calculates the sample standard deviation. On the other hand, stddev calculates the population standard deviation. This is reversed from standard notation and conflicts with the sigmaX and the sX of the Statistics 1 Var app.

Some calculators and statistical packages use PopStdDev and SampStdDev to make it perfectly clear which calulation is being done. P goes with population (division by N) and S goes with sample (division by N-1).

Because I am aware of the frequent confusion about the sample standard deviation as compared with the population standard deviation, I always do a quick check with a small list of values that can be taken as either the population or a sample whenever I start using a new statistical package or app.

Calculating the sample standard deviation should always be greater than calculating the population standard deviation by a factor of sqrt(N/(N-1)) using the same small list.
05-28-2016, 04:31 PM (This post was last modified: 05-28-2016 04:55 PM by salvomic.)
Post: #11
 salvomic Senior Member Posts: 1,365 Joined: Jan 2015
RE: stdevp( ) appears to be mislabeled
(05-28-2016 02:04 PM)Mike Elzinga Wrote:  I'm new to this site.

How does one get Greek and other mathematical symbols? HTML doesn't appear to work in my last post.

In Mac OS X I simply pasted and copied sigma (σ) from "Show symbols" panel, on the top right in the Finder bar)...

otherwise you could use LaTEX (use first \ [ without space and at the end \ ] without spaces and \sigma for the greek small sigma)...
$\sigma$

If you want use it inside a paragraph ($$\sigma$$) use \ ( and \ ) instead...

Attached File(s) Thumbnail(s)

∫aL√0mic (IT9CLU), HP Prime 50g 41CX 71b 42s 12C 15C - DM42 WP34s :: Prime Soft. Lib
05-28-2016, 04:54 PM
Post: #12
 Mike Elzinga Junior Member Posts: 16 Joined: May 2016
RE: stdevp( ) appears to be mislabeled
(05-28-2016 04:31 PM)salvomic Wrote:
(05-28-2016 02:04 PM)Mike Elzinga Wrote:  I'm new to this site.

How does one get Greek and other mathematical symbols? HTML doesn't appear to work in my last post.

In Mac OS X I simply pasted and copied sigma (σ) from "Show symbols" panel, on the top right in the Finder bar)...

otherwise you could use LaTEX (use first \ [ without space and at the end \ ] without spaces and \sigma for the greek small sigma)...
$\sigma$

If you want use it inside a paragraph ($$\sigma$$) use \ ( and \ ) instead...

Thanks. I didn't know LaTEX worked on this site.
05-28-2016, 05:53 PM
Post: #13
 parisse Senior Member Posts: 1,027 Joined: Dec 2013
RE: stdevp( ) appears to be mislabeled
(05-28-2016 03:15 PM)Mike Elzinga Wrote:
(05-28-2016 02:45 PM)parisse Wrote:  I don't see why stddev should correspond to sigmaX and stddevp to sX and not conversely. I find more natural to have the shortest name for the simplest definition and the longest name for the modification required to have an unbiaised estimation. That's why I want to keep stddev for division by sqrt(N). After thinking a bit about the name, stddevs might also be confusing, perhaps stddevpfroms would be better.

Because this is also a pedagogical issue, the names should reflect what calculation is being done.
I see, in fact we disagree because HP made a mistake in the documentation. Xcas documentation says
# stddev Returns the standard deviation of the elements of its argument with an optionnal second argument as pound or the list of standard deviation of the columns of a matrix.
# stddevp Returns an unbiaised estimate of the population standard deviation of the sample (first argument) with an optionnal list of pounds as second argument.
05-28-2016, 06:08 PM
Post: #14
 parisse Senior Member Posts: 1,027 Joined: Dec 2013
RE: stdevp( ) appears to be mislabeled
By the way, it should not be that essential. I mean sqrt(n/n-1) is 1.017... for n=30, in other words your confidence interval will be almost the same for a reasonable sample size (interval length will increase by less than 2%). If you make a poll of size n=1000 we are talking of less than 0.05% change. Insisting too much on the unbiaised vs biaised stddev estimate difference might miss more important comprehension.
05-28-2016, 07:16 PM
Post: #15
 Helge Gabert Senior Member Posts: 460 Joined: Dec 2013
RE: stdevp( ) appears to be mislabeled
Why not just rename stddevp to stddevs - - and everyone should be happy. stddev can stay the way it is.
05-28-2016, 08:12 PM
Post: #16
 Mike Elzinga Junior Member Posts: 16 Joined: May 2016
RE: stdevp( ) appears to be mislabeled
(05-28-2016 07:16 PM)Helge Gabert Wrote:  Why not just rename stddevp to stddevs - - and everyone should be happy. stddev can stay the way it is.

I agree; this would be the easiest fix.

Also change the help for this command to state that it calculates the sample standard deviation. In the help, just change the word "population" to sample.
05-28-2016, 08:58 PM
Post: #17
 Mike Elzinga Junior Member Posts: 16 Joined: May 2016
RE: stdevp( ) appears to be mislabeled
(05-28-2016 06:08 PM)parisse Wrote:  By the way, it should not be that essential. I mean sqrt(n/n-1) is 1.017... for n=30, in other words your confidence interval will be almost the same for a reasonable sample size (interval length will increase by less than 2%). If you make a poll of size n=1000 we are talking of less than 0.05% change. Insisting too much on the unbiaised vs biaised stddev estimate difference might miss more important comprehension.

This is precisely what my little app is demonstrating. It is a pedagogical tool used to show where confidence intervals come from and why we use the language we do in inferential statistics.

1. First the app generates a big population of, say, 1000, with a specified normal distribution. It then checks the actual population mean and population standard deviation and plots the histogram.

2. It allows setting a sample size and takes many (on the order of 100) samples of this size and calculates the standard deviations of the sample means and the standard deviation of the sample standard deviations.

3. If desired, it can then plot a histogram of these and tell us what the standard deviation of all the sample means and standard deviation of all the sample standard deviations are. Several checks with different sample sizes show these histograms get narrower and taller as sample sizes get larger and larger, and the histograms have normal distributions.

3. It also allows incrementing the sample size by a specified amount and repeating this all the way from a small sample size up to some specified large sample size.

4. The sample sizes, the standard deviations of the sample means, and the standard deviations of the sample standard deviations are stored in lists.

5. It then plots the standard deviations of sample means versus sample size, and also the standard deviations of sample standard deviations versus sample size.

6. These plots fit very nicely sigma/sqrt(N) for the sample means, and sigma/sqrt(2N) for the sample standard deviations as expected.

The nice thing about the HP Prime is that it is so fast that it takes very little time to generate all these data and make these plots.
05-29-2016, 05:57 AM
Post: #18
 parisse Senior Member Posts: 1,027 Joined: Dec 2013
RE: stdevp( ) appears to be mislabeled
(05-28-2016 08:12 PM)Mike Elzinga Wrote:
(05-28-2016 07:16 PM)Helge Gabert Wrote:  Why not just rename stddevp to stddevs - - and everyone should be happy. stddev can stay the way it is.

I agree; this would be the easiest fix.

Also change the help for this command to state that it calculates the sample standard deviation. In the help, just change the word "population" to sample.

I'm afraid stddevs is also confusing, stddevpfroms would be better.
05-29-2016, 04:56 PM
Post: #19
 Mike Elzinga Junior Member Posts: 16 Joined: May 2016
RE: stdevp( ) appears to be mislabeled
(05-29-2016 05:57 AM)parisse Wrote:
(05-28-2016 08:12 PM)Mike Elzinga Wrote:  I agree; this would be the easiest fix.

Also change the help for this command to state that it calculates the sample standard deviation. In the help, just change the word "population" to sample.

I'm afraid stddevs is also confusing, stddevpfroms would be better.

05-29-2016, 05:08 PM (This post was last modified: 05-29-2016 05:09 PM by toml_12953.)
Post: #20
 toml_12953 Senior Member Posts: 1,191 Joined: Dec 2013
RE: stdevp( ) appears to be mislabeled
(05-29-2016 04:56 PM)Mike Elzinga Wrote:
(05-29-2016 05:57 AM)parisse Wrote:  I'm afraid stddevs is also confusing, stddevpfroms would be better.

or even

standard_deviation_using_just_a_sample_of_the_total_population_not_the_whole_thing

Tom L

Tom L

If the First Presbyterian Church is nicknamed First Pres,
why isn't the First Methodist Church nicknamed First Meth?
 « Next Oldest | Next Newest »

User(s) browsing this thread: 1 Guest(s)


# What is wrong with my reasoning?

The Questions

70% of all vehicles pass inspection. Assuming vehicles pass or fail independently. What is the probability:

a) exactly one of the next 3 vehicles passes

b) at most 1 of the next 3 vehicles passes

The answer to a) is .189. The way I calculated it was:

P(success) * P(fail) * P(fail) +
P(fail) * P(success) * P(fail) +
P(fail) * P(fail) * P(success) = .7*.3*.3 + .3*.7*.3 + .3*.3*.7 = .189

I summed the 3 possible permutations of 1 success and 2 failures.

For b) the answer is .216. To get that answer you take your answer to a) and add the probability of exactly 0 successes which is P(fail) * P(fail) * P(fail) => .189 + .3*.3*.3 = .216

What I don't understand is why the probability of exactly 0 successes doesn't follow the pattern of exactly 1 success. Why doesn't the "formula" work:

P(fail) * P(fail) * P(fail) +
P(fail) * P(fail) * P(fail) +
P(fail) * P(fail) * P(fail) = .3*.3*.3+.3*.3*.3+.3*.3*.3 = .081

=> .189 + .081 = .27 (not .216)

Now I'm wondering if I calculated the answer to a) the wrong way, and it was merely a coincidence that I got the right answer!

-

The case that the first car passes and the next two fail is different from the case that the first car fails, the second passes, and the third fails. But there is only one case in which all cars fail: namely, all cars fail.

Think of it with coin tosses: if you flip a penny, a nickel, and a dime (different coins, in case you are not in the U.S. or Canada), then there is only one way for all three coins to come out tails, namely, the penny comes out tails, the nickel comes out tails, and the dime comes out tails. But there are three ways in which exactly one coin comes out heads: you can have the penny come out heads and the nickel and dime come out tails; you can have the nickel come out heads and the penny and dime come out tails; or you can have the dime come out heads and the penny and nickel come out tails. These are three different outcomes, but you do not have three different ways in which the three can come out tails.

-

In part a, there are exactly three ways for one out of three cars to pass, they are the three possibilities that you added. But there is only one way for all the cars to fail! The first car must fail, the second car must fail, and the third car must fail. Since there is only one way for this to happen you only consider this one probability.

Btw there are also three ways for exactly two cars to pass and only one way for all three of them to pass. Hope this helps.

-

I'm not too confident with my probability, but I'll try my hand at an explanation.

In the first part, you are considering the next 3 trials as successive events so you can have a success in either the first spot, second spot, or third spot.

In part b, you require all 3 events to be failures so there is only 1 way that the 3 failures can occur in the next 3 events.

Does that make any sense? I never felt probability was my strong point :)

-


# Finding the point on the line of intersection between planes

1. Oct 9, 2016

### Bishamonten

1. The problem statement, all variables and given/known data
Given two planes, P1, P2. Find parametric equations for the line of intersection between the two planes.

2. Relevant equations
P1 = 2x -3y + 4z = 3
P2 = x + 4y - 2z = 7

3. The attempt at a solution
Let N1 be the normal vector to P1, and N2 be the normal vector to P2. Then,

N1 = (2, -3, 4) ; N2 = (1, 4, -2)

Then, the direction numbers for our desired line will come from the components of the cross product between the normal vectors, because it will be parallel to our desired line.

Then, N1xN2 = (-10, 8, 11).

Then, Eq. of line: x = x? -10t ; y = y? +8t ; z = z? + 11t

I currently need to find a point that would be on our desired line, so that I can complete the equation for this line. The book we're using unsatisfactorily just says in it's example version of this type of problem to "let z = 0" in both the given equations of the two planes in it's example, and then solve the system that emerges from that.

My problem is, why? How would I know ahead of time to let one of these three unknowns be 0?(In this case, the back of the book suggests to let y = 0 for my problem here, but I want to know what line of reasoning would let me come to that conclusion).

2. Oct 9, 2016

### andrewkirk

You can solve the problem by setting any one of the coordinates to zero, provided the line is not parallel to but not in the plane where that coordinate is zero. The line will be parallel to such a plane if the corresponding component of the line vector is zero. So you can pick any component for which the corresponding component of the line vector is nonzero. In this case that's any of the three coordinates, because none of the components of the line vector are zero.

What you are doing by setting a coordinate equal to zero is obtaining two equations for the other two coordinates of the point where the line intersects the plane where that coordinate is zero. Those two equations will be linear in the other two coordinates, and hence can be solved to find the values of those two coordinates at the intersection point. Substituting both those values into the equation of either of the planes will allow you to calculate the value of the third coordinate at that point.

Last edited: Oct 9, 2016
3. Oct 9, 2016

### Bishamonten

So I can set any of them to be 0? In this case, when I set z = 0, I get a system in which when solved yields x = 3, y = 1. Substituting these two back into one of the plane equations just tells me that z = 0.

These three values do not match the parametric equation found in the back of the book for this problem, which says that x = 17/4 - 10t ; y = 8t ; z = -11/8 + 11t, so there must be something I am not understanding here.

4. Oct 9, 2016

### andrewkirk

The bit you are not understanding is that parametrisations are not unique. You can pick any point on the line as the 'zero point' of the parametrised line that corresponds to where the parameter value is zero, so there are as many parametrisations as there are points on the line.

Both your parametrisation and the one in the book are correct. To see that, note that the intersection point you found is when $t=0$ in your parametrisation and $t=1/8$ in the book parametrisation.

But your parametrisation is nicer, because it has no fractions in it.

5. Oct 9, 2016

### Bishamonten

Well that's really convenient of parameters. Thank you for clarifying that point to me!

6. Oct 9, 2016

### Ray Vickson

You can use the equations to solve for any two of the variables in terms of the third. For example, you can write the equations as
$$\begin{array}{lcr}2x - 3y &=&3-4z\\ x + 4y &=& 7 + 2z \end{array}$$
This is a simple system of 2 linear equations in 2 unknowns, and is easily solved using elementary algebra, to get
$$x = 3 - \frac{10}{11} z, \; y = 1 + \frac{8}{11} z$$

So, one possible parametrization of the line would be $x = 3 -(10/11)t, y = 1 + ( 8/11) t, z = t$. Of course there are infinitely many points on the intersection line, but one convenient point would be obtained by putting $t = 11$, to eliminate the fractions: $x = -7, y = 9, z = 11$.


# Have you ever used Scratch pad like this?

Find the root of following equation.

$4y^3-2700(1-y)^4=0$

Above equation can be solved using newton raphson iterative method with initial approximation to be unity.

1) First Let $$f(y)=4y^3-2700(1-y)^4$$ We have to find root of above function.

2)Find derivative of $$f(y)$$. Here,$$f^{'}(y)=12y^2+10800(1-y)^3$$

3)Find $$y_n$$ using newton raphson method $y_0=1$ $y_{n+1}=y_n-\frac{f(y_n)}{f^{'}(y_n)}$ $y_{n+1}=y-\frac{4y^3-2700(1-y)^4}{12y^2+10800(1-y)^3}$

You can use calculator or scratchpad to calculate $$y_1,y_2,y_3,....$$.If this equation has real solution ,these values $$y_1,y_2,y_3,....$$ would converge to root of $$f(y)$$.

How to do such iterations smartly?

1) $$\color{Purple}{\text{On first line of scratch pad write y=1}}$$

2) $$\color{Red}{\text{On second line write}}$$

$y=y-\frac{4y^3-2700(1-y)^4}{12y^2+10800(1-y)^3}$

$$\color{Green}{\text{You will see new value of y as result.}}$$

3) $$\color{Orange}{\text{From line 3, continue pasting}}$$

$y=y-\frac{4y^3-2700(1-y)^4}{12y^2+10800(1-y)^3}$

until you get a constant value.If your initial approximation is not too away from actual solution,you would get constant value in no more than 10 steps.

By this method you can solve most of the transcendental equation.

$$\color{blue}{\text{By using scratchpad, try finding sum of initial 10 fibonacci number.}}$$

Note by Aamir Faisal Ansari
3 years, 6 months ago

MarkdownAppears as
*italics* or _italics_ italics
**bold** or __bold__ bold
- bulleted- list
• bulleted
• list
1. numbered2. list
1. numbered
2. list
Note: you must add a full line of space before and after lists for them to show up correctly
paragraph 1paragraph 2

paragraph 1

paragraph 2

[example link](https://brilliant.org)example link
> This is a quote
This is a quote
    # I indented these lines
# 4 spaces, and now they show
# up as a code block.

print "hello world"
# I indented these lines
# 4 spaces, and now they show
# up as a code block.

print "hello world"
MathAppears as
Remember to wrap math in $$...$$ or $...$ to ensure proper formatting.
2 \times 3 $$2 \times 3$$
2^{34} $$2^{34}$$
a_{i-1} $$a_{i-1}$$
\frac{2}{3} $$\frac{2}{3}$$
\sqrt{2} $$\sqrt{2}$$
\sum_{i=1}^3 $$\sum_{i=1}^3$$
\sin \theta $$\sin \theta$$
\boxed{123} $$\boxed{123}$$

Sort by:

Yes !i have used it this way.Sometimes in Recursion problems and like that.

- 3 years, 6 months ago

By using scratchpad, try finding sum of initial 10 fibonacci number.

- 3 years, 6 months ago


# Proving $\phi: V \rightarrow \mathbb{R}^n$ is linear and finding matrix representation of it

Problem: Let $V$ be a $n$-dimensional vectorspace and let $\beta = \left\{v_1, v_2, \ldots, v_n\right\}$ be a basis for $V$. Prove that the coordinate map $\phi_{\beta} : V \rightarrow \mathbb{R}^n$ is linear and determine the matrix $[\phi_{\beta}]_{\beta}^{\gamma}$ with $\gamma$ the standard basis for $\mathbb{R}^n$.

Attempt at solution: Take vectors $v, w \in V$. Then we have \begin{align*} v = \sum_{i=1}^n \lambda_i v_i \qquad \text{and} \qquad w= \sum_{i=1}^n \zeta_i v_i \end{align*} for unique scalars $\lambda_i$ and $\zeta_i \in \mathbb{R}$. It follows that \begin{align*} v+ w &= \sum_{i=1}^n \lambda_i v_i + \sum_{i=1}^n \zeta_i v_i \\ &= \sum_{i=1}^n (\lambda_i + \zeta_i) v_i. \end{align*} Hence we get \begin{align*} \phi_{\beta}(v+w) &= (\lambda_1 + \zeta_1, \lambda_2 + \zeta_2, \ldots, \lambda_n + \zeta_n) \\ &= (\lambda_1, \lambda_2, \ldots, \lambda_n) + (\zeta_1, \zeta_2, \ldots, \zeta_n) \\ &= \phi_{\beta} (v) + \phi_{\beta} (w). \end{align*} Now take a scalar $\mu \in \mathbb{R}$. Then we get \begin{align*} \mu v = (\mu \lambda_1) v_1 + (\mu \lambda_2) v_2 + \ldots + (\mu \lambda_n) v_n, \end{align*} from which it follows that \begin{align*} \phi_{\beta}(\mu v) &= (\mu \lambda_1, \mu \lambda_2, \ldots, \mu \lambda_n) \\ &= \mu (\lambda_1, \lambda_2, \ldots, \lambda_n) \\ &= \mu \phi_{\beta} (v). \end{align*} So $\phi_{\beta} : V \rightarrow \mathbb{R}^n$ is linear.

Then for the matrixrepresentation, I'm not sure what I need to write in the columns of the matrix. If I take a vector $v \in V$, then $\phi_{\beta} (v) = (\lambda_1, \lambda_2, \ldots, \lambda_n)$. And if $\gamma$ is the standard basis for $\mathbb{R}^n$, then this gives $\lambda_1 (1,0,0, \ldots, 0) + \lambda_2 (0,1,0, \ldots, 0) + \ldots + \lambda_n (0,0, \ldots, 1)$. So In the first column just come the coordinates $\lambda_i$? I'm confused about this.

• Could you clarify what "coordinate map" means? – Rory Daulton Jul 3 '15 at 19:12
• @RoryDaulton I think it's pretty clear from his attempted solution. If for $v\in V$ I can write $v=\lambda_1v_1+\dots+\lambda_nv_n$ with respec to basis $\beta$, then the coordinate map would be $\phi_{\beta}(v)=(\lambda_1,\dots,\lambda_n)$ – Alex Mathers Jul 3 '15 at 19:22
• Note that that $[v]_\beta = \phi(v) = [\phi(v)]_\gamma$ – Omnomnomnom Jul 3 '15 at 19:56
• @Omnomnomnom, so what does that mean? Was I right in stating that in the columns are just the coordinates of $v$? – Kamil Jul 3 '15 at 20:00
• @Kamil it means that the $i$th column is the coordinates of $v_i$ with respect to $\beta$. – Omnomnomnom Jul 3 '15 at 20:24

If $T$ is any linear transformation, the matrix $[T]^{\gamma}_{\beta}$ can be written $\begin{bmatrix} [T(v_1)]_{\gamma} & \cdots & [T(v_n)]_{\gamma}\end{bmatrix}$. In other words, the $i$th column of $[T]^{\gamma}_{\beta}$ is $T(v_i)$ written in $\gamma$ coordinates.

For $\phi_{\beta}$, the vector $\phi_{\beta}(v_i)$ is $e_i = (0,\ldots0 ,1,0, \ldots,0)$, so the $i$th column of $[\phi_{\beta}]_{\beta}^{\gamma}$ is $e_i$. This is the same as saying $[\phi_{\beta}]_{\beta}^{\gamma}$ is the identity matrix.

Another way to see this is the equation: $$[\phi_{\beta}]_{\beta}^{\gamma}[v]_{\beta} = [\phi_{\beta}(v)]_{\gamma}$$ But if $v= \lambda_1v_1 + \cdots + \lambda_nv_n$, then we have $[v]_{\beta} = [\phi_{\beta}(v)]_{\gamma} = (\lambda_1,\ldots,\lambda_n)$, and so the matrix $[\phi_{\beta}]_{\beta}^{\gamma}$ is the identity.

Your proof that $\phi_\beta$ is linear is correct.

You're on the right track to compute $[\phi_\beta]_\beta^\gamma$. For a complete computation, write $$\phi_\beta(v_j) = \left(\lambda_{1j},\dotsc,\lambda_{nj}\right) = \lambda_{1j}e_1+\dotsb+\lambda_{nj}e_n$$ for $1\leq j\leq n$. Hence $$[\phi_\beta]_\beta^\gamma = \begin{bmatrix} \lambda_{11} & \cdots & \lambda_{1n} \\ \vdots & \ddots & \vdots \\ \lambda_{n1} & \cdots & \lambda_{nn} \end{bmatrix}$$

Would this not just be the identity matrix? For example, if we write $v=\sum_{i=1}^{n}\lambda_iv_i$ then can equivalently write $v=(\lambda_1,\dots,\lambda_n)_{\beta}$ with respect to the given basis. But if we have $\phi_{\beta}(v)=(\lambda_1,\dots,\lambda_n)\in\mathbb{R}^n$, then we haven't changed it, we've just taken the scalar values. So the matrix would just be the identity.

Unless of course, it wants the "coordinate map" with respect to the standard basis.

• I don't understand your post. What you mean with 'then we haven't changed it'. I don't understand why it is the identity matrix. – Kamil Jul 3 '15 at 19:47


# How can I find the total number of figures in a puzzle where they seem to overlap?

The problem is as follows:

In the following sequence of figures. Which is the total number of right trapezoids which do make up $\textrm{figure 10}$?

The figure is below:

The alternatives given are:

• 66
• 60
• 56
• 64
• 72

What I tried to do is to account for the number of trapezoids:

$\textrm{In figure 1:}$

• There is just $1$

$\textrm{In figure 2:}$

This is some tricky:

I thought that there are $4$

• $2$ make up a small individual unit

• $1$ accounts for the other two

• $1$ accounts for the all of them

$\textrm{In figure 3:}$

• $3$ are consisting of only one figure.

• $2$ are two figures.

• $1$ has three figures.

• $1$ accounts for the three trapezoids in horizontal arrangement and the first oblique bar next to them.

• $1$ accounts for the previous set plus the second oblique bar next to them.

Therefore in the figure are $8$

$\textrm{In figure 4:}$

• $4$ are sets of 1 figure.

• $3$ are sets of 2 figures.

• $2$ are sets of 3 figures.

• $1$ is a set of 4 figures.

• $1$ is a set of the previous 4 figures in horizontal direction plus the first oblique bar appearing next to the trapezoids.

• $1$ is the previous set and the second oblique bar.

• $1$ is the previous set and the third oblique bar.

This accounts for $13$ figures.

So in total the series is comprised of.

$\textrm{1, 4, 8, 13,...}$

From this series I found that the recursion formula is stated as:

$$T_{n}=\frac{1}{2}n^{2}+\frac{3}{2}n-1$$

By replacing with the fourth term it seems to check.

$$T_{4}=\frac{1}{2}\left( 4 \right )^{2}+\frac{3}{2}\left( 4 \right )-1=8+6-1= 13$$

So if what it is being asked is figure number $10$ then by replacing in the previous equation:

$$T_{10}=\frac{1}{2}\left( 10 \right )^{2}+\frac{3}{2}\left( 10 \right )-1=50+15-1= 64$$

Therefore I would choose $64$ as the number of figures in the $\textrm{10th figure}$. I'm not very certain if is it correct?

The answer appears within the alternatives but the process to find the figures was very tedious and it took me a long time to find it, needless to say that to find the recursive formula was another problem as well but eased due the fact which I had some idea of how to find it, as a result.

Is there any other method to speed up the calculations or to find an answer in this situation or to avoid miscounting or double counting the figures?

This looks correct to me. As for the more general question, I would approach the problem by asking 'how can I make up a right trapezoid from these figures?' This breaks naturally into two pieces: trapezoids that use the 'slanting' shapes on the right of the figure, and ones that don't.

If I'm going to use any of the shapes on the right then it's clear I have to use all the ones on the left, and I have to use the ones on the right from left to right; thus, there are as many ways of doing this as there are trapezoids on the right of the figure, i.e. $n-1$.

On the other hand, if I'm using on the sub-trapezoids on the left, then it has to be a contiguous block of them (e.g., the second through the fourth would be valid, but the first and third would not), but any such contiguous block will do; you should be able to see that the number of contiguous blocks is the same as the number of ways of choosing a number $i$ (the 'bottom block') between 1 and $n$ and a number $j$ (the 'top block') between $1$ and $i$; there are exactly $n(n+1)/2$ of these.

So summing up, I get a total of $n(n+1)/2+n-1 = \frac12n^2+\frac32n-1$ possible trapezoids, matching your answer.

• First of when I solved this problem I totally ignored the fact that the figures on the right side were also trapezoids but I was "saved" by the fact the problem asked about counting right trapezoids. Since I'm not good with geometry, do those are also trapezoids?. They look like rectangles which had been tilted to one of their sides, anyway. I'm still lost at where does this $n-1$ comes from. Why not $n-2$?. Most of what I did was by let's say brute force or intuition not really paying attention on details, hence why It took me hours to decode this thing. – Chris Steinbeck Bell Apr 6 '18 at 17:48
• The paragraph where you explain about the subtrapezoid thing I'm stuck at where you mention about $\frac{n(n+1)}{2}$ How do I get to this?. All I know it is that it looks the sum from integers from $1$ to $n$ does this has something to do with the problem?. Maybe if you could rework the explanation a bit better I could understand. – Chris Steinbeck Bell Apr 6 '18 at 17:48

I think you got it.

Counting the stack of $n$ trapezoids in the left part of each diagram is the $n$th triangular number -- normally denoted as $T_n$ but I'll denote it $t_n$ so as not to clash with your notation. The tenth triangular number $t_{10} = 55.$

As additional explanation, let's look at the fourth diagram in your question. Let's label the stack of short, fat trapezoids (looks like a geometrical stack of pancakes) on the left part of that diagram $A,B,C,D$ from top to bottom.

You have $4$ trapezoids with the topmost trapezoid at the top ($A$ by itself, $A+B$, $A+B+C$, and $A+B+C+D$). You'd have $3$ trapezoids with the second-highest trapezoid on top ($B$ by itself, $B+C$, and $B+C+D$). And so on. So the total number of trapezoids, of any height in this stack, is $4+3+2+1 = 10$, which is the fourth triangular number.

Similarly, in the tenth diagram, it would be $10 + 9 + 8 + ... + 2 + 1 = t_{10} = 55$.

A different but completely equivalent way to look at it is that there is $1$ trapezoid that is stacked $10$ high, $2$ trapezoids that are stacked $9$ high, $3$ that are stacked $8$ high ... and $10$ that are stacked $1$ high. It still adds up to $55$.

Since the side pieces extend only the trapezoid that represents the entire stack, you add $n-1$ of those. (From the diagrams you have, the third diagram has two additional trapezoids from the side pieces, hence the $n-1$ formula.)

Adding these two disjoint sets of trapezoids gives your answer $55+(10-1)=64$.

• Okay but how do I know it was a triangular number?. Why is it a triangular number?. Reading solely to the article you mentioned it has gave me an idea but it is not very clear. i.e the second figure has only two right trapezoids in the left, so by calling that triangular number thing there are three figures only? as the second term is three?. Why should be added $n-1$?. Where does it come from?. Can you explain this with more words please? – Chris Steinbeck Bell Apr 6 '18 at 17:33
• Added some more explanation. – John Apr 6 '18 at 17:39
• Maybe I'm too dumb but I can't picture what you just mentioned. $10$ trapezoid with the topmost thin on the top. What does it mean?. I tried to relate their numbers with the triangular numbers again. But it does not check with the fourth term. It says $10$ but in the 4th figure there are $13$. If I ignore some of them and regard the ones in the left plus those in the right would be $4$+$4$ so there are $8$ or Am I understanding it wrong?. I revisited your answer different times but I got tangled up with how should I picture that in my mind, sorry. – Chris Steinbeck Bell Apr 6 '18 at 18:39
• No, you're not too dumb! I tried again. See if that's clearer. – John Apr 6 '18 at 18:58
• Thanks for adding that explanation now I can "see" it better. But, $55$ is not the answer, so I understand that it is required to add something, Why this something to be added has to be $n-1$? you mentioned because it is one unit less of what it is on the stack, so in this portion we do not need to perform the triangular number thing because due the position of those figures with the others makes it possible to know the number by counting them normally (hence getting their total directly) as they will not add up as if they were stacked horizontally?. – Chris Steinbeck Bell Apr 6 '18 at 20:44

ETA: I've added a somewhat shoddy diagram to illustrate what I mean. I've also changed the notation somewhat to be (a bit) more logical.

I'm not sure this is much simpler than what you did, but I numbered the vertices down the left-hand side $B_{0, 0}, B_{1, 0}, \ldots, B_{n, 0}$, then rightward from $B_{0, 0}$ were $B_{0, 1}, B_{0, 2}, \ldots, B_{0, n}$, and rightward from $B_{n, 0}$ were $B_{n, 1}, B_{n, 2}, \ldots, B_{n, n}$. Finally, down the leftmost diagonal, between $B_{0, 1}$ and $B_{n, 1}$, were $B_{1, 1}, B_{2, 1}, \ldots, B_{n-1, 1}$.

Hopefully, it'll be clear that the right trapezoid must contain two vertices of the form $B_{i, 0}$ and $B_{j, 0}$ (without loss of generality, assume $i < j$), since the right angles only live there. The other two vertices must then be $B_{i, k}$ and $B_{j, k}$, for some value of $k$.

Then, there are only two basic classes of right trapezoids:

• If $k = 1$, then $i$ and $j$ can be any values such that $0 \leq i < j \leq n$.

• If $2 \leq k \leq n$, then $i$ and $j$ must be $0$ and $n$, respectively.

There are $\binom{n+1}{2}$ of the former, and $n-1$ of the latter, which gives us a count of

$$\binom{n+1}{2}+n-1 = \frac{n^2+3n-2}{2}$$

which gives us $64$ when $n = 10$, as expected.

• I do not get clearly the idea of counting from the corners. I mean okay, count from the corners but I don't understand the notation you have used. To me this kind of answers would be greatly improved if it had some visual aid as I'm not good with imagination. This binomial $\binom{n+1}{2}$ where does it comes from and why $n-1$?. – Chris Steinbeck Bell Apr 6 '18 at 17:39
• @ChrisSteinbeckBell: I'll add a diagram when I get a chance. The $\binom{n+1}{2}$ examples of the first class come from the fact that $i$ and $j$ can be any two of the integers $0$ through $n$, which is $n+1$ of them. The second class is $n-1$ and not $n$ because the first class already includes the trapezoid $A_0, B_{0, 1}, B_{n, 1}, A_n$, so that $k$ can only range from $2$ through $n$, which is $n-1$ possible values. – Brian Tung Apr 6 '18 at 17:41
• Thanks for the diagram it really eased my understanding but. Where is $j$?. I assume you mean by $i$ the top bottom corner on the left side and $j$ the $1$ seen in the corners starting from $B_{n,1}$?. I can get more or less that $n-1$ is because if in the right side there are $n$ then in the left must be $n-1$ as it is seen in the diagrams. Or is it because of other reason? I'm still confused at how do I know that the total is that specific binomial?. Because is the sum from 1 to n? But if that's the case $n-1$ solely is not the sum of those trapezoids. Can you help me with that? – Chris Steinbeck Bell Apr 6 '18 at 18:32
• @ChrisSteinbeckBell: $i$ and $j$ indicate the upper and lower horizontals of the trapezoid, respectively. Does that help enough? – Brian Tung Apr 6 '18 at 22:33


# Is there a smooth, preferably analytic function that grows faster than any function in the sequence $e^x, e^{e^x}, e^{e^{e^x}}...$

Is there a smooth, preferably analytic function that grows faster than any function in the sequence $$e^x, e^{e^x}, e^{e^{e^x}}$$?

Note: Here the answer is NOT required to be an elementary function, as I know that otherwise the answer would be no.

Edit: Michael has mentioned interpolating a series of functions, but exactly how do I do that in a smooth manner?

• Does this answer your question? A function that grows faster than any function in the sequence $e^x, e^{e^x}, e^{e^{e^x}}$... Jun 9 '20 at 5:21
• This isn't technically a duplicate because I'm not requiring the answer to be an Elementary function. Jun 9 '20 at 5:22
• en.wikipedia.org/wiki/Entire_function#Growth Jun 9 '20 at 5:24
• fair enough, you are also requiring other constraints as well, agreed that this not a duplicate. Jun 9 '20 at 5:32
• You have a sequence of functions $\{f_k(x)\}_{k=1}^{\infty}$ so you can interpolate the points $\{(k, f_k(k))\}_{k=1}^{\infty}$. Jun 9 '20 at 6:02

The link-only comment of metamorphy is actually a full answer, and gives an analytic function, instead of the smooth function of Michael's answer. Instead of hiding behind a link to Wikipedia, I give the construction here with some added detail.

At the end I give a smooth construction as well, and then mention a really cool generalisation I found (Carleman's Theorem.)

## Set-up to apply Wikipedia's construction

First, as in the comments, get a $$C^0$$ increasing function that's faster than all $$\exp^{\circ n} (x):= \overbrace{\exp \big (\exp\big(\dots\exp}^{n \text{ times}}\big(x\big)\big)\big )$$ For instance, you can define $$g(k) := \exp^{\circ k}(k)$$ for naturals $$k\in\mathbb Z_{\ge 1}$$, and then for points in between integers you linearly interpolate, i.e. for $$t\in(0,1)$$, define $$g(k+t):= (1-t) g(k) + t g(k+1).$$ This is faster in the sense that $$g(x)\ge \exp^{\circ n}(x)$$ for all $$x\ge n$$. This is obvious at the integers, and the fact that all $$\exp^{\circ n}(x)$$ are convex proves the result in between the integers (and the fact that $$\exp^{\circ n}(x) \le \exp^{\circ (n+1)}(x)$$). The goal now is to construct an analytic function that beats $$g$$ pointwise. (If you want a smooth function, search for bump functions.)

## Wikipedia's construction

Now Wikipedia in wiki/Entire_function#Growth suggests we define our analytic function as a power series

$$f(z) = g(2)+ \sum_{k=1}^\infty \left(\frac zk\right)^{n_k}$$ where each $$n_k\in 2\mathbb Z_{\ge 1}$$ is chosen so that $$(n_k)$$ is strictly increasing (in particular then $$n_k\ge k$$) and $$\left(\frac{k+1}k\right)^{n_k}>g(k+2).$$

## Proving correctness

First the root test to check its entire: all coefficients $$a_j$$ of the power series $$f(z)=\sum a_j z^j$$ are either 0 or positive, so we have $$\limsup_{j\to\infty} |a_j|^{1/j} = \lim_{k\to\infty} \frac1{k^{n_k/k}} \le \frac1k \to 0.$$ So the radius of convergence is $$1/\limsup |a_j|^{1/j} = \infty$$. Since $$n_k$$ are even we only need to check the behavior for $$x\ge0$$. Now for each $$0\le x\le 2$$, $$f(x)\ge g(2) \ge g(x)$$， and at points $$j+t$$ for $$t\in[0,1), j\ge 2$$, we have $$f(j+t) \ge \left(\frac{j+t}{j-1}\right)^{n_{j-1}} \ge \left(\frac{j}{j-1}\right)^{n_{j-1}} > g(j+1)> g(j+t).$$

## Extra: smooth answer using bump functions

I'll sketch one construction using bump functions here since the OP asked how in an edit. Its probably not the simplest, but I had it lying around for other reasons. Let $$\phi$$ be any smooth, even, non-negative function that is identically one if $$|x|\le 1$$, and zero for $$|x|\ge2$$. Define for $$k\ge 0$$, $$\psi_k(x) := \phi(2^{-k}x) - \phi(2^{-(k+1)} x)$$. Then $$\psi_k$$ is smooth and $$\psi_k$$ is zero outside $$2^{k-1}<|x|<2^{k+1}$$. Define also $$\psi_{-1} = \phi(2^{-1}x)$$. If we also choose $$\phi(x)\le 1$$ , then $$\psi_k\ge 0$$. One checks that for each $$x\in\mathbb R$$, $$\sum_{k\ge -1} \psi_k (x) = 1.$$ In fact, for each $$x$$ only at most 2 of the summands are not zero, and they sum to $$1$$. For instance, $$\phi_{-1}\equiv 1$$ for $$|x|\le 1/2$$, and all other terms are 0. If $$\frac12 < x \le 1$$, then $$\sum_{k\ge -1} \psi_k (x) = \psi_{-1}(x) + \psi_0(x) = \phi(x) = 1,$$ and so on. Now you can just define (remember, for each $$x$$, only 2 summands are nonzero) $$f(x) := \sum_{k\ge-1} \psi_k(x) \exp^{\circ k} (|x|).$$

## Extra 2: Carleman's Theorem

After learning the power series construction above, I also clicked on the Talk page on wikipedia. Apparently something that they were discussing putting into the article is the following great theorem:

Theorem (Carleman) given a complex valued continuous function $$f: \mathbb{R} \rightarrow \mathbb{C}$$ and a strictly positive function $$\epsilon: \mathbb{R} \rightarrow \mathbb{R}_{+}$$, there exists an entire function $$g: \mathbb{C} \rightarrow \mathbb{C}$$ such that $$|f(x)-g(x)|<\epsilon(x)$$ for every $$x \in \mathbb R$$.

This theorem says in particular that there are entire functions that grow at infinity as fast as you want, but also not too fast (i.e. upper and lower bounds on the growth rate), or wobble in some weird way that you precisely prescribe. Basically, any graph you draw, up to as tiny an error you want, with the error improving as $$|x|\to \infty$$, is the graph of an entire function restricted to $$\mathbb R$$. That to me, is crazy!

This result was proven way back in 1927, which is somehow still under copyright protection so I can't link to a free copy (or read it myself, even if I don't understand the language). If you can find it, you can check "Lectures on Complex Approximation" by Dieter Gaier for a short proof taken from a paper of Kaplan, who attributes it to Brelot. The proof is some sort of mixture of the above two ideas, where a lemma is first proven to make up for the fact that you can't use partitions of unity if you want to construct an entire function. Kaplan's paper is free access and is linked below.

• Carleman, T., Sur un théorème de Weierstraß., Arkiv för Mat. B 20, No. 4, 5 p. (1927). ZBL53.0237.02.

• Gaier, Dieter, Lectures on complex approximation. Transl. from the German by Renate McLaughlin, Boston-Basel-Stuttart: Birkhäuser. XV, 196 p.; DM 94.00 (1987). ZBL0612.30003.

• Kaplan, Wilfred, Approximation by entire functions, Mich. Math. J. 3, 43-52 (1956). ZBL0070.06203.
• For a simple solution we can take say $n_k=g(2k)$, giving us $f(z) = 16+ \sum_{k=1}^\infty \left(\frac zk\right)^{ \exp^{\circ 2k}(2k)}$ Jun 11 '20 at 9:31
• @blademan9999 yup, though I believe this is specific to our question's $g$. I made my answer longer(too long? oh well) with two other solutions to the question (a smooth construction, and a reference to a high power theorem) Jun 11 '20 at 11:40
• In Carlemans theorem, it may be good to clarify that $f$ is assumed to be continuous. (For example, the conclusion would not hold for the Dirichlet function.) Jun 11 '20 at 23:09
• @BoazMoerman oops, thank you for the correction! Jun 11 '20 at 23:14

This gives details to my comment: Let $$\{f_k\}_{k=1}^{\infty}$$ be a sequence of functions $$f_k:\mathbb{R}\rightarrow\mathbb{R}$$ that satisfy the following for all $$k \in \{1, 2, 3, ...\}$$:

1. $$f_k(x)>0 \quad \forall x>0$$.

2. $$f_k(x) \leq f_{k+1}(x) \quad \forall x >0$$

3. $$f_k(x)$$ is nondecreasing in $$x$$.

4. $$\lim_{x\rightarrow\infty} \frac{f_{k+1}(x-1)}{f_k(x)} = \infty$$

You can verify that your functions satisfy these properties. Note that: $$f_1(1) \leq f_2(2) \leq f_3(3) \leq f_4(4) \leq ...$$ So we can define $$g:\mathbb{R}\rightarrow\mathbb{R}$$ as any function that is nondecreasing and that smoothly interpolates the points $$\{(k, f_k(k))\}_{k=1}^{\infty}$$.

Then for any positive integer $$m$$ and any $$x >m+1$$ we have: \begin{align} \frac{g(x)}{f_m(x)} &\overset{(a)}{\geq} \frac{g(\lfloor x\rfloor)}{f_m(x)} \\ &= \frac{f_{\lfloor x\rfloor}(\lfloor x\rfloor)}{f_m(x)} \\ &\overset{(b)}{\geq} \frac{f_{m+1}(\lfloor x\rfloor)}{f_m(x)} \\ &\overset{(c)}{\geq} \frac{f_{m+1}(x-1)}{f_m(x)} \end{align} where (a) uses the fact that $$g$$ is nondecreasing; (b) holds because $$\lfloor x\rfloor \geq m+1$$ together with property 2; (c) holds by property 3. Taking a limit as $$x\rightarrow\infty$$ and using property 4 gives $$\lim_{x\rightarrow\infty} \frac{g(x)}{f_m(x)} \geq \lim_{x\rightarrow\infty} \frac{f_{m+1}(x-1)}{f_m(x)} = \infty$$ Thus, $$g$$ grows faster than any of the $$f_m(x)$$ functions.


# How do I find the intersection of a closed 3D curve given by numerical data with a specified plane?

I have a list of 3D data points which form a closed curve. My own data can't be easily generated analytically, but the qualitative behaviour is well represented by a parametric curve families such as {Sin[a1 t] Cos[a2 t], Sin[a3 t] Sin[a4 t], Sin[a5 t]} where the aN are constants. Sample data is given by:

dt = Table[{Sin[3 t] Cos[1 t], Sin[t] Sin[5 t], Sin[2 t]}, {t, -10, 10, 0.005}];
p1 = ListPointPlot3D[dt]

I want to solve for the points where this closed 3D curve cuts a plane I specify but I'm am unable to do it. For example a plane at z = 0.5 as shown here:

p2 = Plot3D[-0.5, {x, -1, 1}, {y, -1, 1}]
Show[p1, p2]

No approach I've tried so far has worked e.g. Interpolation, Nearest to -0.5, use of Select and Drop.

The data does wrap around itself multiple times and getting slightly different solution points on each pass due to differences in the stepsize or something like that is not a problem.

Related I think to this question

In brief: How can I find where a closed curve given by numerical data points cuts a specified plane? Any help you can give is much appreciated.

-
Since it's a closed curve, you could either use periodic splines, periodic Hermite interpolants, or a Fourier fit for your closed curve, and then equate the $z$-component of your curve to the value of your cutting plane. – J. M. Jun 11 '13 at 14:21
Why do you sample over such a broad interval? -Pi to Pi would suffice. – BoLe Jun 11 '13 at 14:51
@BoLe, there's actually a good question in there: how might one know that his data points were sampled over more than one period? – J. M. Jun 11 '13 at 15:25
Related: 10640. Use zeroCrossings[Last /@ dt - 0.5] or zeroCrossings[plane @@@ dt], where plane[x, y, z] == 0 is the equation of the plane, to find the points between which the crossing occurs. – Michael E2 Jun 11 '13 at 19:42
@0x4A4D Thanks for the suggestion and the fast response yesterday. Very useful stuff and it got me quickly into some experimenting that did achieved some success. – fizzics Jun 12 '13 at 8:42

We can use one of the zeroCrossing functions from the answers to this question to construct solutions to where an ordered list of points representing a curve crosses a surface $f(x,y,z)=0$. There are some very nice answers to the linked question with good explanations of the solutions. Use one that returns an interval of indices where the crossing takes place ,such as whuber's answer

zeroCrossings[l_List] := Module[{t, u, v},
t = {Sign[l], Range[Length[l]]} // Transpose; (*List of-1,0,1 only*)
u = Select[t, First[#] != 0 &];               (*Ignore zeros*)
v = Split[u, First[#1] == First[#2] &];       (*Group into runs of+and- values*)
{Most[Max[#[[All, 2]]] & /@ v], Rest[Min[#[[All, 2]]] & /@ v]} // Transpose]

or the SparseArray version in Mr.Wizard's answer, which seems to be quite efficient.

zeroCrossings[l_List] :=
With[{c = SparseArray[l]["AdjacencyLists"]}, {c[[#]], c[[# + 1]]}\[Transpose] &@

The OP's curve points, for which I suppose the corresponding values of t may be lost or unknown:

dt = Table[{Sin[3 t] Cos[1 t], Sin[t] Sin[5 t], Sin[2 t]}, {t, -10, 10, 0.005}];

To help you understand the functions below, I first give the output of zeroCrossings. For the OP's example, we would be interest when z - 0.5 is zero. We translate down the z coordinates of points dt by 0.5 and search for the zero crossings of the translated z coordinates.

zeroCrossings[Last /@ dt - 0.5]

{{168, 169}, {377, 378}, {796, 797}, {1006, 1007}, {1425, 1426},
{1634, 1635}, {2053, 2054}, {2262, 2263}, {2681, 2682}, {2891, 2892},
{3309, 3310}, {3519, 3520}, {3938, 3939}}

Each pair are the indices of points lying on either side of z == 0.5.

Below I use linear interpolation to find the crossing point, which seems simplest. The function zeroPt returns the point where f is zero corresponding to the zero crossing interval crossIDX. The function zeroTime returns the "time", interpolated between the indices of the crossing.

zeroPt[pts_, crossIDX_, f_] /; Length[crossIDX] >= 3 :=
Mean[pts ~Part~ crossIDX[[2 ;; -2]]]; (* unlikely >= 4 - return all zero points? *)
zeroPt[pts_, crossIDX_, f_] /; Length[crossIDX] == 2 :=
(#2 f[#1] - #1 f[#2])/(f[#1] - f[#2]) & @@ pts[[crossIDX]];

zeroTime[pts_, idx_, f_] /; Length[idx] >= 3 := Mean[idx];
zeroTime[pts_, idx_, f_] /; Length[idx] == 2 := First[idx] + Rescale[0., f /@ pts[[idx]]];

### OP's Example

zeroTime[dt, #, Last[#] - 0.5 &] & /@ zeroCrossings[Last /@ dt - 0.5]

{168.405, 377.843, 796.723, 1006.16, 1425.04, 1634.48, 2053.36, 2262.8,
2681.68, 2891.12, 3310., 3519.44, 3938.32}

int = zeroPt[dt, #, Last[#] - 0.5 &] & /@ zeroCrossings[Last /@ dt - 0.5]

{{0.682995, 0.249992, 0.5}, {-0.183003, 0.249985, 0.5}, {0.682998, 0.249994, 0.5},
{-0.183003, 0.249984, 0.5}, {0.68301, 0.249999, 0.5}, {-0.182995, 0.24997, 0.5},
{0.682996, 0.249993, 0.5}, {-0.183001, 0.249981, 0.5}, {0.682997, 0.249993, 0.5},
{-0.183005, 0.249988, 0.5}, {0.683012, 0.25, 0.5}, {-0.182995, 0.249971, 0.5},
{0.682997, 0.249993, 0.5}}

If the curve is an orbit or other periodic trajectory, then the one might be interested in the unique intersections. One can gather the clusters of points with this function and take the Mean to approximate the intersection.

clusters[pts_, dist_] := Gather[pts, EuclideanDistance[##] < dist &];

Example:

Mean /@ clusters[int, 0.001]

{{0.683001, 0.249995, 0.5}, {-0.183, 0.24998, 0.5}}

A plot of the results:

Graphics3D[{
{Opacity[0.1], PointSize[Small], Point @ dt},
{Red, PointSize[Medium], Point[Mean /@ clusters[int, 0.001]]}},
Axes -> True]

### Arbitrary planes

We can apply the method to arbitrary planes. A function plane defining the plane, plane[x, y, z] == 0, is applied to the points and we find where the values cross zero; the function is needed in both zeroCrossings and zeroPt.

Here we will use ConvexHull to show the plane via a polygon containing all the intersection points. We have to project the points onto a plane, which must be chose judiciously so that the projected points are not all collinear, if possible.

Needs["ComputationalGeometry"]

plane[x_, y_, z_] := 2 x - 3 y - 2 z + 0.5;
With[{intersections = zeroPt[dt, #, plane @@ # &] & /@ zeroCrossings[plane @@@ dt]},
Graphics3D[{
{Lighter@Blue, PointSize[Small], Point@dt},
{Darker@Red, PointSize[Large], Point[intersections],
Red, Opacity[0.6],
Polygon[#[[ConvexHull[Most /@ #]]] &[Mean /@ clusters[intersections, 0.001]]]}},
Axes -> True]
]

### Other surfaces

The method works to find where any function of the points crosses zero (in this case surf).

param[u_, v_] := {u v, u, 2 v^2 - 1};
surf[x_, y_, z_] := Evaluate[Subtract @@ Eliminate[{x, y, z} == param[u, v], {u, v}]];
Show[
With[{intersections = zeroPt[dt, #, surf @@ # &] & /@ zeroCrossings[surf @@@ dt]},
Graphics3D[{
{Lighter@Blue, PointSize[Small], Point@dt},
{Red, PointSize[Large],
Point[Mean /@ clusters[intersections, 0.001]]}}, Axes -> True]],
ParametricPlot3D[param[u, v], {u, -1, 1}, {v, -1, 1},
MeshStyle -> Gray, PlotStyle -> Opacity[0.3]]
]

-
+1 Thank you very much for this. The code up to and including the clusters function really solves my problem. The extension to intersections with arbitrary planes and surfaces is also very instructional and useful. – fizzics Jun 13 '13 at 9:04

If you start by interpolating the each component of the curve you can easily solve for a simple plane like z=0.5:

dt = Table[{Sin[3 t] Cos[1 t], Sin[t] Sin[5 t], Sin[2 t]}, {t, -10, 10, 0.005}];
ip = Interpolation[dt[[All, #]]] & /@ {1, 2, 3};
FindRoot[ip[[3]][t] == 0.5, {t, 10}] (* ip[[3]] is the z component of the curve*)
(* {t -> 168.404} *)
#[t /. %] & /@ ip
(* {0.683013, 0.25, 0.5} *)

For other planes you can change the FindRoot criteria, for instance FindRoot[2 ip[[3]][t] + ip[[2]][t] == 0.5, {t, 10}]

-
Thanks for that. I had trouble finding all of the crossings though using FindRoot. I incorporated it into a forward seach style algorithm but I was still missing some points. I also tried to use FindInstance. Thanks for the answer. +1 – fizzics Jun 12 '13 at 9:00

Defining a function P[x_,y_,z_] := a x + b y + c z for suitable constant values a, b, c, the plane is the set of all points P[x, y, z] == 0, i.e. points that satisfy this equation are on the plane. Points for which P[x, y, z] > 0 are on one side of the plane, while those giving P[x, y, z] < 0 are on the other side.

So, the points you want are those on your curve with P[x, y, z] == 0, or an interpolation between any successive points on your curve that give opposite signs for P`.

-


# Determining if a symmetric matrix is positive definite

I have a symmetric matrix where all non-diagonal elements are positive and identical, and all diagonal elements are identical as well. For example, the $3 \times 3$ version of this matrix has the following form: $$\left( \begin{array}{ccc} 2a+b & a & a \\ a & 2a+b & a \\ a & a & 2a+b \end{array} \right)$$

Note that $a>0\ , b>0$. For such a simple form, is there an easy way of determining that the above matrix is positive definite in the general $n \times n$ case? I'd like to show that the matrix is still positive definite when the dimension is higher. Thank you.

• The criterion of Sylvester is well-adapted to your case. – Giuseppe Negro May 24 '18 at 21:37
• @GiuseppeNegro Beat me to it. – KennyB May 24 '18 at 21:53

Yes. Your matrix can be written as $$(a+b)I + a ee^T$$ where $I$ is the identity matrix and $e$ is the vector of ones. This is a sum of a symmetric positive definite (SPD) matrix and a symmetric positive semidefinite matrix. Hence it is SPD.

• Thank you! This is very helpful – SelinH May 24 '18 at 21:57
• It would be cool to actually show that $ee^T$ is positive semidefinite: $x^T e e^T x = (e^Tx)^T(e^Tx)$; since $e^Tx$ is a scalar $(e^T x)^T = e^T x$ whence $x^T e e^T x = (e^T x)^2 \ge 0$. Endorsed! Cheers! – Robert Lewis May 24 '18 at 23:18
• @SelinH You are very welcome. – Carl Christian May 25 '18 at 9:37
• @RobertLewis Thank your for your kind words. I often find it difficult to select the right level of detail. – Carl Christian May 25 '18 at 9:43

In addition to Carl's answer (+1) you can use the Gershgorin circle theorem which is one I'll often try when I see diagonally dominant matrices like this one.

For each row, the sum of the absolute values of the off-diagonal entries is $2a$. The theorem then says that every eigenvalue is in a closed disc of radius $2a$ centered at the diagonal elements $2a + b$, so for any eigenvalue $\lambda$ we have $\lambda \geq b > 0$.

$$\begin{bmatrix} 2a+b & a & a\\ a & 2a+b & a\\ a & a & 2a+b\end{bmatrix} = (a+b) \, \mathrm I_3 + a \, 1_3 1_3^\top = (a+b) \, \mathrm I_3 + 3a \left(\frac{\, 1_3}{\sqrt{3}}\right) \left(\frac{\, 1_3}{\sqrt{3}}\right)^\top$$

Hence, the eigenvalues are $4a + b$ and $a+b$, with multiplicities $1$ and $2$, respectively. Since $a, b > 0$, both eigenvalues are positive, i.e., the given symmetric matrix is positive definite.


NumPy provides a mechanism for performing mathematical operations on arrays of unequal shapes:

>>> import numpy as np

# a shape-(3, 4) array
>>> x = np.array([[-0. , -0.1, -0.2, -0.3],
...               [-0.4, -0.5, -0.6, -0.7],
...               [-0.8, -0.9, -1. , -1.1]])

# a shape-(4,) array
>>> y = np.array([1, 2, 3, 4])

# multiplying a shape-(4,) array with a shape-(3, 4) array
# y is multiplied by each row of x
>>> x * y
array([[-0. , -0.2, -0.6, -1.2],
[-0.4, -1. , -1.8, -2.8],
[-0.8, -1.8, -3. , -4.4]])

In effect, NumPy treated y as if its contents had been broadcasted along a new dimension, such that y was a shape-(3, 4) 2D array, which makes it compatible for multiplying with x:

$$\left( \begin{array}{*{3}{X}} -0.0 & -0.1 & -0.2 & -0.3 \\ -0.4 & -0.5 & -0.6 & -0.7 \\ -0.8 & -0.9 & -1.0 & -1.1 \end{array} \right) % \cdot \left( \begin{array}{*{3}{X}} 1 & 2 & 3 & 4 \end{array}\right) % \rightarrow \left( \begin{array}{*{3}{X}} -0.0 & -0.1 & -0.2 & -0.3 \\ -0.4 & -0.5 & -0.6 & -0.7 \\ -0.8 & -0.9 & -1.0 & -1.1 \end{array} \right) % \cdot \left( \begin{array}{*{3}{X}} 1 & 2 & 3 & 4 \\ 1 & 2 & 3 & 4 \\ 1 & 2 & 3 & 4 \end{array}\right) \$$

It is important to note that NumPy doesn’t really create this broadcasted version of y behind the scenes; it is able to do the necessary computations without having to redundantly copy its contents into a shape-(3,4) array. Doing so would be a waste of memory and computation. That being said, this replication process conveys exactly the mathematics of broadcast operations between arrays; thus the preceding diagram reflects how you should always envision broadcasting.

Broadcasting is not reserved for operations between 1-D and 2-D arrays, and furthermore both arrays in an operation may undergo broadcasting. That being said, not all pairs of arrays are broadcast-compatible.

# Broadcast multiplications between a
# shape-(3, 1, 2) array and a shape-(3, 1)
# array.
>>> x = np.array([[[0, 1]],
...
...               [[2, 3]],
...
...               [[4, 5]]])

>>> y = np.array([[ 0],
...               [ 1],
...               [-1]])

# shape-(3, 1, 2) broadcast-multiply with
# shape-(3, 1) produces shape-(3, 3, 2)
>>> x * y
array([[[ 0,  0],
[ 0,  1],
[ 0, -1]],

[[ 0,  0],
[ 2,  3],
[-2, -3]],

[[ 0,  0],
[ 4,  5],
[-4, -5]]])

# an example of broadcast-incompatible arrays
# a shape-(2,) array with a shape-(3,) array
>>> np.array([1, 2]) * np.array([0, 1, 2])
ValueError: operands could not be broadcast together with shapes (2,) (3,)

Array Broadcasting is a mechanism used by NumPy to permit vectorized mathematical operations between arrays of unequal, but compatible shapes. Specifically, an array will be treated as if its contents have been replicated along the appropriate dimensions, such that the shape of this new, higher-dimensional array suits the mathematical operation being performed.

We will now summarize the rules that determine if two arrays are broadcast-compatible with one another, and what the shape of the resulting array will be after the mathematical operation between the two arrays is performed.

Array broadcasting cannot accommodate arbitrary combinations of array shapes. For example, a (7,5)-shape array is incompatible with a shape-(11,3) array. Trying to add two such arrays would produce a ValueError. The following rules determine if two arrays are broadcast-compatible:

To determine if two arrays are broadcast-compatible, align the entries of their shapes such that their trailing dimensions are aligned, and then check that each pair of aligned dimensions satisfy either of the following conditions:

• the aligned dimensions have the same size
• one of the dimensions has a size of 1

The two arrays are broadcast-compatible if either of these conditions are satisfied for each pair of aligned dimensions.

Note that it is okay to have one array with a higher-dimensionality and thus to have “dangling” leading dimensions. Any size-1 dimension or “missing” dimension will be filled-in by broadcasting the content of that array.

Considering the example from the preceding subsection, let’s see that the shape-(4,3) and shape-(3,) arrays satisfy these rules for broadcast-compatibility:

     array-1: 4 x 3
array-2:     3
result-shape: 4 x 3

Let’s look an assortment of pairs of array-shapes and see whether or not they are broadcast-compatible:

     array-1:         8
array-2: 5 x 2 x 8
result-shape: 5 x 2 x 8

array-1:     5 x 2
array-2: 5 x 4 x 2
result-shape: INCOMPATIBLE

array-1:     4 x 2
array-2: 5 x 4 x 2
result-shape: 5 x 4 x 2

array-1: 8 x 1 x 3
array-2: 8 x 5 x 3
result-shape: 8 x 5 x 3

array-1: 5 x 1 x 3 x 2
array-2:     9 x 1 x 2
result-shape: 5 x 9 x 3 x 2

array-1: 1 x 3 x 2
array-2:     8 x 2
result-shape: INCOMPATIBLE

array-1: 2 x 1
array-2:     1
result-shape: 2 x 1

NumPy provides the function broadcast_to, which can be used to broadcast an array to a specified shape. This can help us build our intuition for broadcasting. Let’s broadcast a shape-(3,4) array to a shape-(2,3,4) array:

# Demonstrating np.broadcast_to
>>> x = np.array([[ 0,  1,  2,  3],
...               [ 4,  5,  6,  7],
...               [ 8,  9, 10, 11]])

# Explicitly broadcast a shape-(3,4) array
# to a shape-(2,3,4) array.
array([[[ 0,  1,  2,  3],
[ 4,  5,  6,  7],
[ 8,  9, 10, 11]],

[[ 0,  1,  2,  3],
[ 4,  5,  6,  7],
[ 8,  9, 10, 11]]])

Given the following pairs of array-shapes, determine what the resulting broadcasted shapes will be. Indicate if a pair is broadcast-incompatible.

1. 7 x 2 with 7
2. 4 with 3 x 4
3. 1 x 3 x 1 with 8 x 1 x 1
4. 9 x 2 x 5 with 2 x 5
5. 3 with 3 x 3 x 2

## A Simple Application of Array Broadcasting¶

Here we provide a simple real-world example where broadcasting is useful. Suppose you have a grade book for 6 students, each of whom have taken 3 exams; naturally, you store these scores in a shape-(6,3) array:

# grades for 6 students who have taken 3 exams
# axis-0 (rows):    student
# axis-1 (columns): exams
>>> import numpy as np
>>> grades = np.array([[ 0.79,  0.84,  0.84],
...                    [ 0.87,  0.93,  0.78],
...                    [ 0.77,  1.00,  0.87],
...                    [ 0.66,  0.75,  0.82],
...                    [ 0.84,  0.89,  0.76],
...                    [ 0.83,  0.71,  0.85]])

We might be interested to see how each of these scores compare to the mean score for that specific exam. Based on our discussion from the last section, we can easily compute the mean-score for each exam (rounded to 2 decimal places):

# compute the mean score for each exam (rounded to 2 decimal places)
>>> mean_exam_scores = np.round(mean_exam_scores, 2)
>>> mean_exam_scores
array([ 0.79,  0.85,  0.82])

grades is a shape-(6,3) array and mean_exam_scores is a shape-(3,) array, and we want to compute the offset of each exam score from its respective mean. At first glance, it seems like we will have to loop over each row of our grades array and subtract from it the mean_exam_scores, to compute the offset of each exam score from the respective mean-score:

# Using a for-loop to compute score offsets.

# Shape-(6,3) array that will store (score - mean) for each
# exam score.

# iterates over each row of grades
# scores_per_student is a shape-(3,) array of exam scores
# for a given student. This matches the shape of
# mean_exam_scores, thus we can perform this subtraction
score_offset[n] = scores_per_student - mean_exam_scores

Given our discussion of vectorized operations from the last section, you should recoil at the sight of a for-loop in code that is performing array-arithmetic. We might as well get out our abacuses and spreadsheets at this point. Fortunately, we can make use of broadcasting to compute these offsets in a concise, vectorized way:

# Using broadcasting to subtract a shape-(3,) array
# from a shape-(6,3) array.
>>> score_offset = grades - mean_exam_scores
>>> score_offset
array([[ 0.  , -0.01,  0.02],
[ 0.08,  0.08, -0.04],
[-0.02,  0.15,  0.05],
[-0.13, -0.1 ,  0.  ],
[ 0.05,  0.04, -0.06],
[ 0.04, -0.14,  0.03]])

According to the broadcasting rules detailed above, when you invoke grades - mean_exam_scores, NumPy will recognize that mean_exam_scores has the same shape as each row of grades and thus it will apply the subtraction operation on each row of grades with mean_exam_scores. In effect, the content of mean_exam_scores has been broadcasted to fill a shape-(6,3) array, so that the element-wise subtraction can be performed. Again, we emphasize that NumPy doesn’t actually unnecessarily replicate the data of mean_exam_scores, and that this model of broadcasting merely conveys the mathematical process that is transpiring.

Generate a random array of 10,000 2D points using np.random.rand. Compute the “center of mass” of these points, which is simply the average x-coordinate and the average y-coordinate of these 10,000 points. Then, use broadcasting to compute the shape-(10000,2) array that stores the position of the points relative to the center of mass. For example, if the center of mass is $$(0.5, 1)$$, and the absolute position of a point is $$(2, 3)$$, then the position of that point relative to the center of mass is simply $$(2, 3) - (0.5, 1) = (1.5, 2)$$

## Size-1 Axes & The newaxis Object¶

### Inserting Size-1 Dimensions into An Array¶

As conveyed by the broadcasting rules, dimensions of size-1 are special in that they can be broadcasted to any size. Here we will learn about introducing size-1 dimensions into an array, for the purpose of tailoring its shape for broadcasting.

You can introduce size-1 dimensions to an array without changing the overall size (i.e. total number of entries in an array. Thus we are free to add size-1 dimensions to an array via the reshape function. Let’s reshape a shape-(3,) array into a shape-(1, 3, 1, 1) array:

>>> import numpy as np

# Reshaping an array to introduce size-1 dimensions.
# The size of the array is 3, regardless of introducing
# these extra size-1 dimensions.
>>> np.array([1, 2, 3]).reshape(1, 3, 1, 1)
array([[[[1]],

[[2]],

[[3]]]])

Thus the 1-D array with three entries has been reshaped to possess 4-dimensions: “one stack of three sheets, each containing a single row and column”. There is another way to introduce size-1 dimensions. NumPy provides the newaxis object for this purpose. Let’s immediately demonstrate how np.newaxis can be used:

# demonstrating the usage of the numpy.newaxis object
>>> x= np.array([1, 2, 3])
>>> y= x[np.newaxis, :, np.newaxis, np.newaxis]
>>> y
array([[[[1]],

[[2]],

[[3]]]])
>>> y.shape
(1, 3, 1, 1)

Indexing x as x[np.newaxis, :, np.newaxis, np.newaxis] returns a “view” of x as a 4D array with size-1 dimensions inserted as axes 0, 2, and 3. The resulting array is not a copy of x; it points to the exact same data as x, but merely with a different indexing layout. This is no different than what we achieved via reshaping: x.reshape(1, 3, 1, 1).

### Utilizing Size-1 Dimensions for Broadcasting¶

Moving on to a more pressing matter: why would we ever want to introduce these spurious dimensions into an array? Let’s take an example to demonstrate the utility of size-1 dimensions.

Suppose that we want to multiply all possible pairs of entries between two arrays: array([1, 2, 3]) with array([4, 5, 6, 7]). That is, we want to perform twelve multiplications, and have access to each result. At first glance, combining a shape-(3,) array with a shape-(4,) array seems inadmissible for broadcasting; we seem to be doomed to perform nested for-loops like a bunch of cavemen and cavewomen. Fortunately, we can make clever use of size-1 dimensions so that we can perform this computation in a vectorized way.

Let’s introduce size-1 dimensions into x:

# Inserting size-1 dimensions into x and y in
>>> x_1d = np.array([1, 2, 3])
>>> x = x_1d.reshape(3, 1)
>>> x
array([[1],
[2],
[3]])

>>> y = np.array([4, 5, 6, 7])

x is now a shape-(3, 1) array and y is a shape-(4,) array. According to the broadcasting rules, these arrays are broadcast-compatible and will multiply to produce a shape-(3, 4) array. Let’s see that multiplying these two arrays will exactly produce the twelve numbers that we are after:

# broadcast-multiplying x and y
>>> x * y
array([[ 4,  5,  6,  7],
[ 8, 10, 12, 14],
[12, 15, 18, 21]])

$$\left( \begin{array}{*{1}{X}} 1 \\ 2 \\ 3 \end{array} \right) % \cdot \left( \begin{array}{*{4}{X}} 4 & 5 & 6 & 7 \end{array}\right) % \rightarrow \left( \begin{array}{*{4}{X}} 1 & 1 & 1 & 1 \\ 2 & 2 & 2 & 2 \\ 3 & 3 & 3 & 3 \end{array}\right) % \cdot \left( \begin{array}{*{4}{X}} 4 & 5 & 6 & 7 \\ 4 & 5 & 6 & 7 \\ 4 & 5 & 6 & 7 \end{array}\right) % = \left( \begin{array}{*{4}{X}} 1\cdot4 & 1\cdot5 & 1\cdot6 & 1\cdot7 \\ 2\cdot4 & 2\cdot5 & 2\cdot6 & 2\cdot7 \\ 3\cdot4 & 3\cdot5 & 3\cdot6 & 3\cdot7 \end{array}\right) \$$

See that entry (i, j) of the resulting array corresponds to x_1d[i] * y[j].

Through the use of simple reshaping, shrewdly inserting size-1 dimensions allowed us to coerce NumPy into performing exactly the combination-multiplication that we desired. Furthermore, a keen understanding of what broadcasting is provides us with a clear interpretation of the structure of the result of this calculation. That is, if I reshape x to be a shape-$$(M, 1)$$ array, and y is a shape-$$(N,)$$ array, then (according to broadcasting rules) x * y would produce a shape-$$(M, N)$$ array storing the product of each of x’s $$M$$ numbers with each of y’s $$N$$ numbers.

Given the shape-(2,3,4) array:

>>> x = np.array([[[ 0,  1,  2,  3],
...                [ 4,  5,  6,  7],
...                [ 8,  9, 10, 11]],
...
...               [[12, 13, 14, 15],
...                [16, 17, 18, 19],
...                [20, 21, 22, 23]]])

Normalize x such that each of its rows, within each sheet, will sum to a value of 1. Make use of the sequential function np.sum, which should be called only once, and broadcast-division.

A digital image is simply an array of numbers, which instructs a grid of pixels on a monitor to shine light of specific colors, according to the numerical values in that array.

An RGB-image can thus be stored as a 3D NumPy array of shape-$$(V, H, 3)$$. $$V$$ is the number of pixels along the vertical direction, $$H$$ is the number of pixels along the horizontal, and the size-3 dimension stores the red, blue, and green color values for a given pixel. Thus a $$(32, 32, 3)$$ array would be a 32x32 RBG image.

You often work with a collection of images. Suppose we want to store N images in a single array; thus we now consider a 4D shape-(N, V, H, 3) array. For the sake of convenience, let’s simply generate a 4D-array of random numbers as a placeholder for real image data. We will generate 500, 48x48 RGB images:

>>> images = np.random.rand(500, 48, 48, 3)

Using the sequential function np.max and broadcasting, normalize images such that the largest value within each color-channel of each image is 1.

We will conclude this section by demonstrating an important, non-trivial example of array broadcasting. Here, we will find that the most straightforward use of broadcasting is not necessarily the right solution for our problem, and we will see that it can be important to first refactor the mathematical approach taken to perform a calculation before using broadcasting. Specifically, we will see that our initial approach for using of broadcasting is memory-inefficient.

Suppose we have two, 2D arrays. x has a shape of $$(M, D)$$ and y has a shape of $$(N, D)$$. We want to compute the Euclidean distance (a.k.a. the $$L_2$$-distance) between each pair of rows between the two arrays. That is, if a given row of x is represented by $$D$$ numbers $$(x_0, x_1, \ldots, x_{D-1})$$, and similarly, a row y is represented by $$(y_0, y_1, \ldots, y_{D-1})$$, and we want to compute the Euclidean distance between the two rows:

$$\sqrt{(x_{0} - y_{0})^2 + (x_{1} - y_{1})^2 + \ldots + (x_{D-1} - y_{D-1})^2} = \sqrt{\sum_{i=0}^{D-1}{(x_{i} - y_{i})^2}}$$

Doing this for each pair of rows should produce a total of $$M\times N$$ distances. The previous subsection stepped us through a very similar calculation, albeit with lower-dimensional arrays. Let’s proceed by performing this computation in three different ways:

1. Using explicit for-loops
3. Refactoring the problem and then using broadcasting

For the sake of being concrete, we will compute all of the pairwise Euclidean distances between the rows of these two arrays:

# a shape-(5, 3) array
>>> x = np.array([[ 8.54,  1.54,  8.12],
...               [ 3.13,  8.76,  5.29],
...               [ 7.73,  6.71,  1.31],
...               [ 6.44,  9.64,  8.44],
...               [ 7.27,  8.42,  5.27]])

# a shape-(6, 3) array
>>> y = np.array([[ 8.65,  0.27,  4.67],
...               [ 7.73,  7.26,  1.95],
...               [ 1.27,  7.27,  3.59],
...               [ 4.05,  5.16,  3.53],
...               [ 4.77,  6.48,  8.01],
...               [ 7.85,  6.68,  6.13]])

Thus we will want to compute a total of 30 distances, one for each pair of rows from x and y.

### Pairwise Distances Using For-Loops¶

Performing this computation using for-loops proceeds as follows:

def pairwise_dists_looped(x, y):
"""  Computing pairwise distances using for-loops

Parameters
----------
x : numpy.ndarray, shape=(M, D)
y : numpy.ndarray, shape=(N, D)

Returns
-------
numpy.ndarray, shape=(M, N)
The Euclidean distance between each pair of
rows between x and y."""
# dists[i, j] will store the Euclidean
# distance between  x[i] and y[j]
dists = np.empty((5, 6))

for i, row_x in enumerate(x):     # loops over rows of x
for j, row_y in enumerate(y): # loops over rows of y
# Subtract corresponding entries of the rows,
# squares each difference, and then sums them. This
# exactly matches our equation for Euclidean
# distance (we will do the square root later)
dists[i, j] = np.sum((row_x - row_y)**2)

# we still need to take the square root of
# each of our numbers
return np.sqrt(dists)

Be sure to step through this code and see that dists stores each pair of Euclidean distances between the rows of x and y.

### Pairwise Distances Using Broadcasting (Unoptimized)¶

Now, let’s use of vectorization to perform this distance computation. It must be established immediately that the method that we about to develop here is memory-inefficient. We will address this issue in detail at the end of this subsection.

We start off our vectorized computation by shrewdly inserting size-1 dimensions into x and y, so that we can perform $$M \times N$$ subtractions between their pairs of length-$$D$$ rows. This creates a shape-$$(M, N, D)$$ array.

# subtract shape-(5, 1, 3) with shape-(1, 6, 3)
# produces shape-(5, 6, 3)
>>> diffs = x.reshape(5, 1, 3) - y.reshape(1, 6, 3)
>>> diffs.shape
(5, 6, 3)

It is important to see, via broadcasting, that diffs[i, j] stores x[i] - y[j]. Thus we need to square each entry of diffs, sum over its last axis, and take the square root, in order to produce our $$M \times N$$ Euclidean distances:

# producing the Euclidean distances
>>> dists = np.sqrt(np.sum(diffs**2, axis=2))
>>> dists.shape
(5, 6)

Voilà! We have produced the distances in a vectorized way. Let’s write this out formally as a function:

def pairwise_dists_crude(x, y):
"""  Computing pairwise distances using vectorization.

Parameters
----------
x : numpy.ndarray, shape=(M, D)
y : numpy.ndarray, shape=(N, D)

Returns
-------
numpy.ndarray, shape=(M, N)
The Euclidean distance between each pair of
rows between x and y."""
# The use of np.newaxis here is equivalent to our
# use of the reshape function
return np.sqrt(np.sum((x[:, np.newaxis] - y[np.newaxis])**2, axis=2))

Regrettably, there is a glaring issue with the vectorized computation that we just performed. Consider the largest sized array that is created in the for-loop computation, compared to that of this vectorized computation. The for-loop version need only create a shape-$$(M, N)$$ array, whereas the vectorized computation creates an intermediate array (i.e. diffs) of shape-$$(M, N, D)$$. This intermediate array is even created in the one-line version of the code. This will create a massive array if $$D$$ is a large number!

Suppose, for instance, that you are finding the Euclidean between pairs of RGB images that each have a resolution of $$32 \times 32$$ (in order to see if the images resemble one another). Thus in this scenario, each image is comprised of $$D = 32 \times 32 \times 3 = 3072$$ numbers ($$32^2$$ pixels, and each pixel has 3 values: a red, blue, and green-color value). Computing all the distances between a stack of 5000 images with a stack of 100 images would form an intermediate array of shape-$$(5000, 100, 3072)$$. Even though this large array only exists temporarily, it would have to consume over 6GB of RAM! The for-loop version requires $$\frac{1}{3027}$$ as much memory (about 2MB).

Is our goose cooked? Are we doomed to pick between either slow for-loops, or a memory-inefficient use of vectorization? No! We can refactor the mathematical form of the Euclidean distance in order to avoid the creation of that bloated intermediate array.

### Optimized Pairwise Distances¶

Performing the pairwise subtraction between the respective rows of x and y is what created the over-sized intermediate array in our previous calculation. Thus we want to rewrite the Euclidean distance equation such that none of the terms require broadcasting beyond the size of $$M \times N$$.

The Euclidean distance equation, ignoring the square root for now, can be refactored by multiplying out each squared term as so:

$$\sum_{i=0}^{D-1}{(x_{i} - y_{i})^2} = \sum_{i=0}^{D-1}{x_{i}^2} + \sum_{i=0}^{D-1}{y_{i}^2} - 2\sum_{i=0}^{D-1}{x_{i} y_{i}}$$

Keep in mind that we must compute this for each pair of rows in x and y. We will find that this formulation permits the use of matrix multiplication, such that we can avoid forming the shape-$$(M, N, D)$$ intermediate array.

The first two terms in this equation are straight-forward to calculate, and, when combined, will only produce a shape-$$(M, N)$$ array. For both x and y, we square each element in the array and then sum over the columns for each row:

# Computing the first two terms of the
# refactored Euclidean distance equation

# creates a shape-(5,) array
>>> x_sqrd_summed = np.sum(x**2, axis=1)

# creates a shape-(6,) array
>>> y_sqrd_summed = np.sum(y**2, axis=1)

We must insert a size-1 dimension in x so that we can add all pairs of numbers between the resulting shape-$$(M, 1)$$ and shape-$$(N,)$$ arrays. This will compute $$\sum_{i=0}^{D-1}{x_{i}^2} + \sum_{i=0}^{D-1}{y_{i}^2}$$ for all of the $$M \times N$$ pairs of rows:

# add a shape-(5, 1) array with a shape-(6, ) array
# to create a shape-(5, 6) array
>>> x_y_sqrd = x_sqrd_summed[:, np.newaxis] + y_sqrd_summed
>>> x_y_sqrd.shape
(5, 6)

This leaves the third term to be computed. It is left to the reader to show that computing this sum of products for each pair of rows in x and y is equivalent to performing the matrix multiplication $$-2\;(x \cdot y^{T})$$, where y has been transposed so that it has a shape of $$(D, N)$$. This matrix multiplication of a shape-$$(M, D)$$ array with a shape-$$(D, N)$$ array produces a shape-$$(M, N)$$ array. Therefore, we can compute this final term without needing to create a larger, intermediate array.

Thus the third term in our equation, $$-2\sum_{i=0}^{D-1}{x_{i} y_{i}}$$, for all $$M \times N$$ pairs of rows, is:

# computing the third term in the distance
# equation, for all pairs of rows
>>> x_y_prod = -2 * np.matmul(x, y.T)  # np.dot can also be used to the same effect
>>> x_y_prod.shape
(5, 6)

Having accounted for all three terms, we can finally compute the Euclidean distances:

# computing all the distances
>>> dists = np.sqrt(x_y_sqrd + x_y_prod)
>>> dists.shape
(5, 6)

In total, we have successfully used vectorization to compute the all pairs of distances, while only requiring an array of shape-$$(M, N)$$ to do so! This is the memory-efficient, vectorized form - the stuff that dreams are made of. Let’s write the function that performs this computation in full.

def pairwise_dists(x, y):
""" Computing pairwise distances using memory-efficient
vectorization.

Parameters
----------
x : numpy.ndarray, shape=(M, D)
y : numpy.ndarray, shape=(N, D)

Returns
-------
numpy.ndarray, shape=(M, N)
The Euclidean distance between each pair of
rows between x and y."""
dists = -2 * np.matmul(x, y.T)
dists +=  np.sum(x**2, axis=1)[:, np.newaxis]
dists += np.sum(y**2, axis=1)
return  np.sqrt(dists)

Takeaway:

The specific form of an equation can have a major impact on the memory-footprint of its vectorized implementation in NumPy. This issue can be safely overlooked in cases where you can be certain that the array shapes at play will not lead to substantial memory consumption. Otherwise, take care to study the form of the equation, to see if it can be recast in a way that alleviates its memory-consumption bottlenecks.

Reading Comprehension: Checking the equivalence of the three pairwise distance functions

Use the function numpy.allclose to verify that the three methods for computing the pairwise distances produce the same numerical results.

Generating the random array of 10,000, 2D points, and their “center-of-mass”.

# find the mean x-coord and y-coord of the 10000 points
>>> pts = np.random.rand(10000, 2)
>>> center_of_mass = pts.mean(axis=0)  # -> array([mean_x, mean_y])
>>> center_of_mass.shape
(2,)

# Use broadcasting to compute the position of each point relative
# to the center of mass. The center of mass coordinates are subtracted
# from each of the 10000 points, via broadcast-subtraction
>>> relative_pos = pts - center_of_mass # shape-(10000,2) w/ shape-(2,)
>>> relative_pos.shape
(10000, 2)

1. Incompatible
2. 3 x 4
3. 8 x 3 x 1
4. 9 x 2 x 5
5. Incompatible

# a shape-(2, 3, 4) array
>>> x = np.array([[[ 0,  1,  2,  3],
...                [ 4,  5,  6,  7],
...                [ 8,  9, 10, 11]],
...
...               [[12, 13, 14, 15],
...                [16, 17, 18, 19],
...                [20, 21, 22, 23]]])

# sum over each row, within a sheet
>>> summed_rows = x.sum(axis=1)
>>> summed_rows
array([[12, 15, 18, 21],
[48, 51, 54, 57]])

# this shape-(2,4) array can be broadcast-divided
# along the rows of x, if we insert a size-1 axis
>>> x_norm = x / summed_rows[:, np.newaxis, :]

# verifying the solution
>>> x_norm.sum(axis=1)
array([[ 1.,  1.,  1.,  1.],
[ 1.,  1.,  1.,  1.]])

# a collection of 500 48x48 RGB images
>>> images = np.random.rand(500, 48, 48, 3)

# finding the max-value within each color-channel of each image
>>> max_vals = images.max(axis=(1,2))
>>> max_vals.shape
(500, 3)

# we can insert size-1 dimensions so that we can
# the pixels of the images.
# broadcasting (500, 48, 48, 3) with (500, 1, 1, 3)
>>> normed_images = images / max_vals.reshape(500, 1, 1, 3)

# checking that all the max-values are 1
>>> normed_images.max(axis=(1,2))
array([[ 1.,  1.,  1.],
[ 1.,  1.,  1.],
[ 1.,  1.,  1.],
...,
[ 1.,  1.,  1.],
[ 1.,  1.,  1.],
[ 1.,  1.,  1.]])

# a rigorous check
>>> np.all(normed_images.max(axis=(1,2)) == 1)
True

Checking the equivalence of the three pairwise distance functions: Solution

numpy.allclose returns True if all pairwise elements between two arrays are almost-equal to one another.

>>> x = np.array([[ 8.54,  1.54,  8.12],
...               [ 3.13,  8.76,  5.29],
...               [ 7.73,  6.71,  1.31],
...               [ 6.44,  9.64,  8.44],
...               [ 7.27,  8.42,  5.27]])

>>> y = np.array([[ 8.65,  0.27,  4.67],
...               [ 7.73,  7.26,  1.95],
...               [ 1.27,  7.27,  3.59],
...               [ 4.05,  5.16,  3.53],
...               [ 4.77,  6.48,  8.01],
...               [ 7.85,  6.68,  6.13]])

>>> np.allclose(pairwise_dists_looped(x, y), pairwise_dists_crude(x, y))
True

>>> np.allclose(pairwise_dists_crude(x, y), pairwise_dists(x, y))
True


# GATE2003-62

8.5k views

In a permutation $a_1 ... a_n$, of $n$ distinct integers, an inversion is a pair $(a_i, a_j)$ such that $i < j$ and $a_i > a_j$.

What would be the worst case time complexity of the Insertion Sort algorithm, if the inputs are restricted to permutations of $1. . . n$ with at most $n$ inversions?

1. $\Theta(n^2)$
2. $\Theta(n\log n)$
3. $\Theta(n^{1.5})$
4. $\Theta(n)$

edited
0
what is inversion???????????????????????
0
Let A[1..N] be an array of distinct elements.
If i < j and A[i]>A[j] the pair i,j is known as inversion of the Array.
0

As the question says, how the Inversion is defined .

In a permutation $a_1 ... a_n$, of n distinct integers, an inversion is a pair $(a_i, a_j)$ such that $i < j$ and $a_i > a_j$.

• One important thing to see is Difference between swap and Inversions.
• Swapping is done explicitly by the programmer, hence a explicit feature whereas, Inversion is an implicit feature which is defined in the input .

Ex :- Take the input => $\left \{ 0,1,9,8,10,7,6,11 \right \}$

How many Inversions here : $\left \{ 9,8 \right \}$ , $\left \{ 9,7 \right \}$ , $\left \{ 9,6 \right \}$ ,$\left \{ 8,7 \right \}$ , $\left \{ 8,6 \right \}$ ,  $\left \{ 10,7 \right \}$ and $\left \{ 10,6 \right \}$. Hence, it is an implicit feature of the input and not any explicit operation done (Swap) .

Actual Time complexity of Insertion Sort is defined as $\Theta \left ( N + f(N) \right )$, where $f\left ( N \right )$ is the total number of Inversions .

Ex :- Again take the input => $\left \{ 0,6,7,8,9,10,1 \right \}$

Here, how many comparisons will be there to place $1$ in the right place ?

• First of all, $1$ is compared with $10$ - Returns True as it is smaller than $10$.
• Then, with $9$ - again True.
• Similar, happens with $6,7,8$ - All return True .

Hence, There $5$ comparisons were the Inversions and $1$ more comparison will be there, in which outer while loop fails .

For, placing $1$ in the right place $\bf{6}$ comparisons are there .

Hence, Total Comparisons in the Insertion sort will be :- Total number of elements for which our while loop fails + Total number of inversions in the input

• Total number of elements for which our while loop fails :- Suppose the input $\left \{ 0,6,7,8,9,10,1 \right \}$. Here, first $0$ will be kept as it is and one while loop fail comparison for each element, hence total comparisons like this :- $\left ( N-1 \right )=O\left ( N \right )$ comparisons.
• Total number of inversions in the input :- Best Case : $0$ and in the Worst Case : $\frac{N\left ( N-1 \right )}{2} = O\left ( N^{2} \right )$

Total Time complexity of insertion sort : $\Theta (N+f(N))$

It is given here that at most $N$ inversions are there, so we get the best Time complexity :- $\Theta (N+f(N)) = \Theta (N + N) = \Theta (N)$ .

Correct Answer: $D$

edited
1
Excellent explanation Sir!
0
0
In worst case when the array is sorted in descending order the complexity would be O(n^2). But since its restricted to n inversions, so the complexity would be now O(n).
1
Restriction on inversión to n mean array is almost sorted in increasing order.let us take an example

5 1 2 3 4

Here insertion sort will stop after first pass by making array sorted and inversion will be number of swap
0

@Kapil sir you saved my many hours the key is just read the question carefully : they explicit mentioned that number of inversions are restricted up to n then why we considering O(n^2) case it should be done only in O(n) time.

0

The link mentioned is not working.

https://gateoverflow.in/253812/gatebook-2019-ds3-9
0
could u please elaborate more on "Actual Time complexity of Insertion Sort is defined as Θ(N+f(N))Θ(N+f(N)), where f(N)f(N) is the total number of Inversions ." ??? I didn't get this point
0
The number of comparisons in insertion sort are O($N^{2}$) then how is the total time complexity O(N)??

1
I think (7,6) should also be an inversion.

ANSWER: D. $\Theta(n)$

REASON:

Count of number of times the inner loop of insertion sort executes is actually equal to number of inversions in input permutation $a_1,a_2,\dots a_n$. Since for each value of $i = 1... n$, $j$ take the value $1... i-1$, which means for every $j<i$ it checks if $a[j] > a[i]$.

In any given permutation, maximum number of inversions possible is $n(n-1)/2$ which is $O(n^2)$. It is the case where the array is sorted in reverse order. Therefore, to resolve all inversions i.e., worst case time complexity of insertion sort is $\Theta(n^2)$.

However, as per the question the number of inversion in input array is restricted to $n$. The worst case time complexity of insertion sort reduces to $\Theta(n)$.

INSERTION SORT ALGORITHM (for reference)

edited by
0
array could be 5,4,3,2,1

in best case of insertion sort will take O(n)

But how in worst case O(n) ?
13

If the array elements are 5,4,3,2,1..then number of inversion equals 10 which is twice the size of array. Question restrict the number of inversions to $n$.

Best case of insertion sort means that the inner while loop was never executed at all. That will happen when there is no inversion at all (array is sorted). Since, the outer loop has traversed the element of array once therefore the complexity is O(n) in best case.

However in this particular case we should not make a claim that the inner loop will not be executed. Actually it will execute, but only O(n) time in total (not just for some iteration of outer loop). So in worst case i.e., irrespective of whatever is the given permutation, as long as number of inversion in that permutation is $n$ insertion sort will sort that array in O(n). That actually the claim here.

0
means in 5,4,3,2,1

4 inversion it will be 1,5,4,3,2

then 1 inversion 1,5,4,2,3....We have to stop here
6
@srestha inversions are not operations (swaps) done by us- it is a property of the input. 5,4,3,2,1 has 4+3+2+1 = 10 inversions.
0
ok , so here atmost n inversion . that is why complexity O(n).rt?

As we know in insertion sort complexity depend on number of inversions rt?
4
yes, exactly. This question is about analysing the complexity of insertion sort.
3
Outer loop executes n times. Inner loop will also be executed n times as there are n inversions.

So complexity will be $\Theta$ (n + n) . So $\Theta$ (n)

http://www.geeksforgeeks.org/time-complexity-insertion-sort-inversions
0
please add that running time is Θ(n+d), where n we get from outer loop and d is the  number of inversion.

here number of inversion=n

so time complexity=Θ(n+n)= Θ(n)
0
inversions are not operations (swaps) done by us- it is a property of the input. 5,4,3,2,1 has 4+3+2+1 = 10 inversions.
@Arjun
i can't get it sir .

Ans is D: ⊝(n),

Insertion sort runs in Θ(n + f(n)) time, where f(n) denotes the number of inversion initially present in the array being sorted.

1 vote
You can learn this point:-

Insertion sort complexity is

(N+ no. of inversion).
1 vote

Just take a simple case and examine:

Usually, the worst case corresponds to whenever input is in decreasing order, so

let Input = 5,4,3,2,1

Now, we need to make sure the inversion property applies i.e. n=5 inversions are only possible

For the given I, there's 10 inversions. With the number 5, there's 4 inversions - <5,4> , <5,3>, <5,2>, <5,1>. We just need 1 more. By rearranging, let I = 5,1,2,4,3. Now there's 5 inversions. We have a perfect example close to worst case.

Now apply the insertion sort Algo to I.

Iteration(i,j)  Input

0,0 - 5,1,2,4,3

1,0 - 1,5,2,4,3

2,0 - 1,2,5,4,3

3,0 - 1,2,4,5,3

4,0 - 1,2,4,3,5

4,1 - 1,2,3,4,5

Total Iterations = 5. Therefore Theta(n), option D

0

@arjuno

can you explain little more how you find total iteration and what is that 1,0  2,0 ....4,0  4,1  stands for?

1 vote

Time complexity of insertion sort is: $O(n + i)$ where $i$ is the number of inversions.

In the worst case, there are $n(n-1)/2=O(n^2)$ inversions, and hence the Time Complexity of Insertion Sort degrades to $O(n+n^2)=O(n^2)$.

Here, in the worst case we're restricted to $n$ inversions, so Time Complexity = $\theta(n+n)=\theta(n)$

Option D

inputs are restricted to permutations of $1...n$ with at most $n$ inversions

so if we take input as $n,1,2….n-1,n-2$

here no. of inversions is$n$

e.g. if n=9

$9,1,2,3,4,5,6,8,7$

inversions are

$(9,1)(9,2)(9,3)(9,4)(9,5)(9,6)(9,8)(9,7)(8,7) i.e. 9$

so worst case input according to question is  $n,1,2….n-1,n-2$

this will take O(N) time to sort using insertion sort as except n & n-2 all elements are sorted
ago

## Related questions

1
7.3k views
In a permutation $a_1 ... a_n$, of n distinct integers, an inversion is a pair $(a_i, a_j)$ such that $i < j$ and $a_i > a_j$. If all permutations are equally likely, what is the expected number of inversions in a randomly chosen permutation of $1. . . n$? $\frac{n(n-1)}{2}$ $\frac{n(n-1)}{4}$ $\frac{n(n+1)}{4}$ $2n[\log_2n]$
The unusual $\Theta(n^2)$ implementation of Insertion Sort to sort an array uses linear search to identify the position where an element is to be inserted into the already sorted part of the array. If, instead, we use binary search to identify the position, the worst case running time will remain $\Theta(n^2)$ become $\Theta(n (\log n)^2)$ become $\Theta(n \log n)$ become $\Theta(n)$
In the following $C$ program fragment, $j$, $k$, $n$ and TwoLog_n are integer variables, and $A$ is an array of integers. The variable $n$ is initialized to an integer $\geqslant 3$, and TwoLog_n is initialized to the value of $2^*\lceil \log_2(n) \rceil$ for (k = 3; k <= n; k++) A[k] ... $\left\{m \mid m \leq n, \text{m is prime} \right\}$ { }
Let $G= (V,E)$ be a directed graph with $n$ vertices. A path from $v_i$ to $v_j$ in $G$ is a sequence of vertices ($v_{i},v_{i+1}, \dots , v_j$) such that $(v_k, v_{k+1}) \in E$ for all $k$ in $i$ through $j-1$. A simple path is a path in which no vertex ... longest path length from $j$ to $k$ If there exists a path from $j$ to $k$, every simple path from $j$ to $k$ contains at most $A[j,k]$ edges