Get old tweets by user using tweepy

I am trying to gather the tweets of a user navalny, from 01.11.2017 to 31.01.2018 using tweepy. I have ids of the first and last tweets that I need, so I tried the following code:

import tweepy

consumer_key = '' 
consumer_secret = ''
access_token = ''
access_token_secret = ''

auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)

api = tweepy.API(auth)

t = api.user_timeline(screen_name='navalny', since_id = 933000445307518976, max_id = 936533580481814529)

However, the returned value is an empty list.

  • What is the problem here?

  • Are there any restrictions on the history of tweets that I can get?

  • What are possible solutions?

1 answer

  • answered 2018-03-22 13:02 Justin

    Quick answer: Using Tweepy you can only retrieve the last 3200 tweets from the Twitter REST API for a given user. Unfortunately the tweets you are trying to access are older than this.

    Detailed answer: I did a check using the code below:

    import tweepy
    from tweepy import OAuthHandler
    
    
    def tweet_check(user):
        """
        Scrapes a users most recent tweets
        """
        # API keys and initial configuration
        consumer_key = ""
        consumer_secret = ""
        access_token = ""
        access_secret = ""
        # Configure authentication
        authorisation = OAuthHandler(consumer_key, consumer_secret)
        authorisation.set_access_token(access_token, access_secret)
        api = tweepy.API(authorisation)
        # Requests most recent tweets from a users timeline
        tweets = api.user_timeline(screen_name=user, count=2, 
                                   max_id=936533580481814529)
        for tweet in tweets:
            tid = tweet.id
            print(tid)
    
    
    twitter_users = ["@navalny"]
    
    for twitter_user in twitter_users:
        tweet_check(twitter_user)
    

    This test returns nothing before 936533580481814529

    Using a seperate script I scraped all 3200 tweets, the max Twitter will let you scrape and the youngest tweet id I can find is 943856915536326662

    Seems like you have run into Twitter's tweet scraping limit for user timelines here.