R: Get tweets data on statuses identified via search query.

search_tweets {rtweet} | R Documentation

## Get tweets data on statuses identified via search query.

### Description

Returns Twitter statuses matching a user provided search query. ONLY RETURNS DATA FROM THE PAST 6-9 DAYS. To return more than 18,000 statuses in a single call, set "retryonratelimit" to TRUE.

search_tweets2 Passes all arguments to search_tweets. Returns data from one OR MORE search queries.

### Usage

    search_tweets(
      q,
      n = 100,
      type = "recent",
      include_rts = TRUE,
      geocode = NULL,
      max_id = NULL,
      parse = TRUE,
      token = NULL,
      retryonratelimit = FALSE,
      verbose = TRUE,
      ...
    )

    search_tweets2(...)

### Arguments

`q` | Query to be searched, used to filter and select tweets to return from Twitter's REST API. Must be a character string not to exceed maximum of 500 characters. Spaces behave like boolean "AND" operator. To search for tweets containing at least one of multiple possible terms, separate each search term with spaces and "OR" (in caps). For example, the search `q = "data science"` looks for tweets containing both "data" and "science" located anywhere in the tweets and in any order. When "OR" is entered between search terms, `query = "data OR science"`, Twitter's REST API should return any tweet that contains either "data" or "science." It is also possible to search for exact phrases using double quotes. To do this, either wrap single quotes around a search query using double quotes, e.g., `q = '"data science"'` or escape each internal double quote with a single backslash, e.g., `q = "\"data science\""`. Some other useful query tips: Exclude retweets via `"-filter:retweets"` Exclude quotes via `"-filter:quote"` Exclude replies via `"-filter:replies"` Filter (return only) verified via `"filter:verified"` Exclude verified via `"-filter:verified"` Get everything (firehose for free) via `"-filter:verified OR filter:verified"` Filter (return only) tweets with links to news articles via `"filter:news"` Filter (return only) tweets with media `"filter:media"`
`n` | Integer, specifying the total number of desired tweets to return. Defaults to 100. Maximum number of tweets returned from a single token is 18,000. To return more than 18,000 tweets, users are encouraged to set `retryonratelimit` to TRUE. See details for more information.
`type` | Character string specifying which type of search results to return from Twitter's REST API. The current default is `type = "recent"`, other valid types include `type = "mixed"` and `type = "popular"`.
`include_rts` | Logical, indicating whether to include retweets in search results. Retweets are classified as any tweet generated by Twitter's built-in "retweet" (recycle arrows) function. These are distinct from quotes (retweets with additional text provided from sender) or manual retweets (old school method of manually entering "RT" into the text of one's tweets).
`geocode` | Geographical limiter of the template "latitude,longitude,radius" e.g., `geocode = "37.78,-122.40,1mi"`.
`max_id` | Character, returns results with an ID less than (that is, older than) or equal to 'max_id'. Especially useful for large data returns that require multiple iterations interrupted by user time constraints. For searches exceeding 18,000 tweets, users are encouraged to take advantage of rtweet's internal automation procedures for waiting on rate limits by setting `retryonratelimit` argument to TRUE. It some cases, it is possible that due to processing time and rate limits, retrieving several million tweets can take several hours or even multiple days. In these cases, it would likely be useful to leverage `retryonratelimit` for sets of tweets and `max_id` to allow results to continue where previous efforts left off.
`parse` | Logical, indicating whether to return parsed data.frame, if true, or nested list, if false. By default, `parse = TRUE` saves users from the wreck of time and frustration associated with disentangling the nasty nested list returned from Twitter's API. As Twitter's APIs are subject to change, this argument would be especially useful when changes to Twitter's APIs affect performance of internal parsers. Setting `parse = FALSE` also ensures the maximum amount of possible information is returned. By default, the rtweet parse process returns nearly all bits of information returned from Twitter. However, users may occasionally encounter new or omitted variables. In these rare cases, the nested list object will be the only way to access these variables.
`token` | Every user should have their own Oauth (Twitter API) token. By default `token = NULL` this function looks for the path to a saved Twitter token via environment variables (which is what 'create_token()' sets up by default during initial token creation). For instruction on how to create a Twitter token see the tokens vignette, i.e., 'vignettes("auth", "rtweet")' or see `?tokens`.
`retryonratelimit` | Logical indicating whether to wait and retry when rate limited. This argument is only relevant if the desired return (n) exceeds the remaining limit of available requests (assuming no other searches have been conducted in the past 15 minutes, this limit is 18,000 tweets). Defaults to false. Set to TRUE to automate process of conducting big searches (i.e., n > 18000). For many search queries, esp. specific or specialized searches, there won't be more than 18,000 tweets to return. But for broad, generic, or popular topics, the total number of tweets within the REST window of time (7-10 days) can easily reach the millions.
`verbose` | Logical, indicating whether or not to include output processing/retrieval messages. Defaults to TRUE. For larger searches, messages include rough estimates for time remaining between searches. It should be noted, however, that these time estimates only describe the amount of time between searches and not the total time remaining. For large searches conducted with `retryonratelimit` set to TRUE, the estimated retrieval time can be estimated by dividing the number of requested tweets by 18,000 and then multiplying the quotient by 15 (token reset time, in minutes).
`...` | Further arguments passed as query parameters in request sent to Twitter's REST API. To return only English language tweets, for example, use `lang = "en"`. For more options see Twitter's API documentation.

### Details

Twitter API documentation recommends limiting searches to 10 keywords and operators. Complex queries may also produce API errors preventing recovery of information related to the query. It should also be noted Twitter's search API does not consist of an index of all Tweets. At the time of searching, the search API index includes between only 6-9 days of Tweets.

Number of tweets returned will often be less than what was specified by the user. This can happen because (a) the search query did not return many results (the search pool is already thinned out from the population of tweets to begin with), (b) because user hitting rate limit for a given token, or (c) of recent activity (either more tweets, which affect pagination in returned results or deletion of tweets). To return more than 18,000 tweets in a single call, users must set `retryonratelimit` argument to true. This method relies on updating the `max_id` parameter and waiting for token rate limits to refresh between searches. As a result, it is possible to search for 50,000, 100,000, or even 10,000,000 tweets, but these searches can take hours or even days. At these durations, it would not be uncommon for connections to timeout. Users are instead encouraged to breakup data retrieval into smaller chunks by leveraging `retryonratelimit` and then using the status_id of the oldest tweet as the `max_id` to resume searching where the previous efforts left off.

### Value

List object with tweets and users each returned as a data frame.

A tbl data frame with additional "query" column.

### See Also

https://developer.twitter.com/en/docs/tweets/search/api-reference/get-search-tweets

Other tweets: `get_favorites()`, `get_mentions()`, `get_my_timeline()`, `get_timeline()`, `lists_statuses()`, `lookup_statuses()`, `tweets_data()`, `tweets_with_users()`

### Examples

    ## Not run:

    ## search for 1000 tweets mentioning Hillary Clinton
    hrc <- search_tweets(q = "hillaryclinton", n = 1000)

    ## data frame where each observation (row) is a different tweet
    hrc

    ## users data also retrieved. can access it via users_data()
    users_data(hrc)

    ## search for 1000 tweets in English
    djt <- search_tweets(q = "realdonaldtrump", n = 1000, lang = "en")

    ## preview tweets data
    djt

    ## preview users data
    users_data(djt)

    ## exclude retweets
    rt <- search_tweets("rstats", n = 500, include_rts = FALSE)

    ## perform search for lots of tweets
    rt <- search_tweets(
      "trump OR president OR potus", n = 100000,
      retryonratelimit = TRUE
    )

    ## plot time series of tweets frequency
    ts_plot(rt, by = "mins")

    ## make multiple independent search queries
    ds <- Map(
      "search_tweets",
      c("\"data science\"", "rstats OR python"),
      n = 1000
    )

    ## bind by row whilst preserving users data
    ds <- do_call_rbind(ds)

    ## preview tweets data
    ds

    ## preview users data
    users_data(ds)

    ## End(Not run)

    ## Not run:

    ## search using multilple queries
    st2 <- search_tweets2(
      c("\"data science\"", "rstats OR python"),
      n = 500
    )

    ## preview tweets data
    st2

    ## preview users data
    users_data(st2)

    ## check breakdown of results by search query
    table(st2$query)

    ## End(Not run)

[Package rtweet version 0.7.0 Index]