updating graph labels in matplotlib

Using Matplotlib animation I have come across the issue where I can update data point positions but for some reason not labels. The function.

latitude, longitude, size, labels = coordinates(sheet, i)

gives me 4 lists and the first 2 function as intended returning me only active points . however I cannot assign a label that updates after each iteration so they stay around when they shouldn't. How do I set a label to refresh? any ideas? Sorry for the rough code.I'm using matplotlib, basemap and numpy for this.

x, y = map(0,0)
labels = (' ')
point = map.plot(x, y, 'ro', markersize = 5) [0]

label = plt.text(x,y,labels)

def init():
    point.set_data([], [])
    #label.set_text = ([])
    return (point, label,)

# animation function.  This is called sequentially
def animate(i):
    latitude, longitude, size, labels = coordinates(sheet, i)
    x, y = map(longitude, latitude)
    point.set_data(x, y)
    label.set_text(labels)
    return (label, point,)

# call the animator.  blit=True means only re-draw the parts that have changed.
anim = animation.FuncAnimation(plt.gcf(), animate, init_func=init,
                               frames=24, interval=1000, blit=True)

plt.show()