Fill pandas Dataframe columns of different length with a loop

This is an example of what I would like to do:

history=Dataframe()

range= 40
for k in range (range):

    epochs=k+1
    loss=np.random.rand(k+1,1)


history[k]=loss

This fills the Dataframe with k columns, with the loss array.

The problem is that, for the following iteration, lossis bigger than in the previous iteration, and bigger in the following.

So, there is a conflict between the dataframe index:

raise ValueError('Length of values does not match length of ' 'index')
ValueError: Length of values does not match length of index

Is there any way to fill the dataframe as I want without this problem?

If the dataframe wasn't filled with a loop, I would think of filling it with NaNs.

1 answer

  • answered 2017-06-17 19:52 Wen

    Is this what you want ?

    l= 3
    for k in range(l):
    
        epochs=k+1
        loss=np.random.rand(k+1,1)
        if k ==0:
            history=pd.DataFrame(loss)
        else:
            history=pd.concat([history,pd.DataFrame(loss)],axis=1)
    history.columns=range(3)
    
    history
              0         1         2
    0  0.043321  0.882806  0.578819
    1       NaN  0.216240  0.558565
    2       NaN       NaN  0.739184