代写data编程、Python编程设计代做

日期：2023-10-26 08:09

Assignment 1

Deadline: 10.24.2023 - 11.59pm

By the deadline, you are required to submit the following:

Report. Prepare a single PDF file that presents clear and concise evidence of

your completion of each of the listed tasks. Use the overleaf template available on iCorsi.

Source code. Create a single zipped Python script using the template available on iCorsi, performing each of the specified tasks. If a task is accomplished

in the code but not documented in the report, it will be considered incomplete.

Therefore, make sure to thoroughly report all your work in the submitted report. Jupyter notebook files will not be accepted.

General hints. The question marked with * is more challenging, and we recommend possibly leaving them as the last questions to solve. To obtain the

maximum grade, not only should all exercises be completed correctly, but the

plots must also be clear, have a legend, and have labels on the axes and the

text should be written in clear English. For saving plots in high quality, consider using the command matplotlib.pyplot.savefig. Clarity of plots

and text will account in total for 5 points.

Submission rules: As already discussed in class, we will stick to the following rules.

? Submit ONE pdf file containing your report and ONE (zipped) .py file

containing the code. Use the templates and name your files

NAME SURNAME.pdf and NAME SURNAME.py. We will read and

correct only these files. Other files present in the folder will not be read.

If such files are missing, they will be evaluated with a score of 0.

? Code either not written in Python or not using PyTorch receives a grade

of 0. Of course you can use auxiliary packages when needed (matplotlib,

numpy,. . . ), but for the learning part, you must use PyTorch.

? Submission between 00.00am - 00.10 of Wednesday 25th will be accepted

with no reduction.

? If plagiarism is suspected, TAs and I will thoroughly investigate the situation, and we will summon the student for a face-to-face clarification

regarding certain answers they provided. In case of plagiarism, a score

1

reduction will be applied to all the people involved, depending on their

level of involvement.

? If extensive usage of AI tools is detected, we will summon the student

for a face-to-face clarification regarding certain answers they provided.

If the answers are not adequately supported with in-person answers, we

will proceed to apply a penalty to the evaluation, ranging from 10% to

100%.

Polynomial regression (with gradient descent) [95 points]

Let z ∈ R and consider the polynomial

p(z) = z

4

100

? z

3 + z

2 ? 10z =

X

4

k=1

z

kwk (1)

where w = [w1, w2, w3, w4]

T = [?10, 1, ?1,

1

100 ]

T

. This polynomial can be also

expressed as the dot product of two vectors, namely

p(z) = wT x x = [z, z2

, z3

, z4

]

T

(2)

Consider an independent and identically distributed (i.i.d.) dataset D = {(zi

, yi)}

N

i=1,

where yi = p(zi) + εi

, and each εi

is drawn from a normal distribution with

mean zero and standard deviation σ.

Now, assuming that the vector w is unknown, linear regression could estimate it using the dot-product form presented in Equation 2. To achieve this we

can move to another dataset

D

′

:= {(xi

, yi)}

N

i=1 x

i = [zi

, z2

i

, z3

i

, z4

i

]

T

The task of this assignment is to perform polynomial regression using gradient descent with PyTorch, even if a closed-form solution exists.

1. (5 pts) Define a function

plot_polynomial(coeffs, z_range, color=’b’)

Where coeffs is a np.array containing [w0, w1, w2, w3, w4]

T

(w0 in this

case is equal to 0) z_range is the interval [zmin, zmax] of the z variable;

color represent a color. Use the function to plot the polynomial. Report

and comment on the plot.

2. (10 pts) Write a function

create_dataset(coeffs, z_range, sample_size, sigma, seed=42)

that generates the dataset D′

. Here coeffs is a np.array containing

[w0, w1, w2, w3, w4]

T

, z_range is the interval [zmin, zmax] of the z variable;

sample_size is the dimension of the sample; sigma is the standard

deviation of the normal distrbution from which εi are sampled; seed is

the seed for the random procedure.

2

3. (5 pts) Use the code of the previous point to generate data with the following parameters

? Each zi should be in the interval [?3, 3]

? σ = 0.5

? Use a sample size of 500 for training data and a seed of 0

? Use a sample size of 500 for evaluation data and a seed of 1

4. (5 pts) Define a function

visualize_data(X, y, coeffs, z_range, title="")

that plot the polynomial p(z) and the generated data, (train and evaluation), where X, y are as returned from the function create_dataset,

coeffs are the coefficient of the polynomial, z_range is the interval

[zmin, zmax] of the z variable and title may be helpful to distinguish between the training and the evaluation plots. What we mean with this

question, is the following: Provide two plots containing both the true

polynomial; In one, add a scatter plot with the training data, and in the

other a scatter plot with the evaluation data. Use the function to visualize

the data. Report and comment on the plots.

5. (20 pts) Perform polynomial regression on D using linear regression on

D′ and reports some comments on the training procedure. Consider your

training procedure good when the training loss is less than 0.5. This

works in my code with a number of steps equal to 3000. In particular,

explain:

? How you preprocessed the data.

? Which learning rate do you use, and why. What happens if the

learning rate is too small; what happens if the learning rate is too

high, and why.

? If bias should be set as True of False in torch.nn.Linear and

why.

You should use the data you generated at point 3. Note: If you plan to reuse code explained during the lecture, explicitly indicate which parts are

being reused or readjusted/modified. In any case, the steps must always

be explained and understood in detail.

6. (10 pts) Plot the training and evaluation loss as functions of the iterations

and report them in the same plot. If you use both steps and epochs, you

can choose either of the two, as long as it is clear from the plot and the

plot reports what we expect - namely, that the loss functions decrease.

7. (5 pts) Plot the polynomial you got with your estimated coefficient as

well as the original one in the same plot.

3

8. (10 pts) Plot the value of the parameters at each iteration as well as the

true value. What we expect is a plot having on the x axis the number of

steps/epochs and on the y axis the value of the parameters while they

are learned. Then, in the same graph, plot a horizontal line with the true

value.

9. (15 pts) Re-train the model with the following parameters:

? Each zi should be in the interval [?3, 3]

? σ = 0.5

? Use a sample size of 10 for training data and a seed of 0

? Use a sample size of 500 for evaluation data and a seed of 1

? Keep the learning rate you chose at the previous point.

Report: A plot with both the training and evaluation loss as functions of

the iterations in the same plot and the polynomial you got with your estimated coefficient as well as the original one in the same plot. Comment

on what is going on. Note: Do not overwrite the code, keep the original

training and this new one as two separate parts.

10. (5 pts*) This part is less trivial and should be considered as a separate

exercise. Suppose you have to learn the function

f(x) = 5 sin(x) + 3

and you know that your data are x

i belong to the interval [?a, a]. Which

performances would you expect using linear regression in the following

cases?

(a) a = 0.01

(b) a = 5

Note: We want to see some code that generates noisy data from f, do the

linear regression and we want to see the loss value on it. If you know

the mathematics behind it, you can use your knowledge to answer and

validate the data, but as this is a Lab exam, we want to see empirical

evidence.

Questions [5 points]

Please answer these questions in max 10 lines. In Lecture 3, we have seen an

explicit step of the parameter update for one-dimensional linear regression,

namely

wt+1 = wt ? α

?L

?w (wt, bt) (3)

bt+1 = bt ? α

?L

?b (wt, bt) (4)

Hint: Do we have any information about the gradient of the loss function with

respect to the parameter at the very beginning of the training loop?

4

(a) Do you think that could be beneficial to choose two different values of α

(learning rate) for Equation (3) and (4)?

(b) Explain the difference between choosing two different learning rates in

principle and what is instead done by the adaptive methods - e.g, Adagrad. If you don’t know the method, you can study one of the following

resources:

? Section 8.5.1 of Deep Learning Book

? Section 12.7 of Dive Into Deep Learning

5