Sunday, 28 February 2016

Crowdsourced feeds from ThreatCrowd

Voting

Voting was added to ThreatCrowd recently, and I've been pleased to see a number of users regularly contributing votes.


These votes provide a useful source of malicious indicators, and so I've now put these into a feed in two files:

 https://www.threatcrowd.org/feeds/domains.txt
 https://www.threatcrowd.org/feeds/ips.txt
https://www.threatcrowd.org/feeds/hashes.txt

These feeds are not a substitute for the scale of auto-extracted command and control domains or the quality of some commercially provided feeds. But crowd-sourcing does go some way towards the quick sharing of threat intelligence between the community.

Updates
These files are updated once per hour, on the hour.

API
You can submit votes via the interface, or a simple API:

This will place a vote for "good.com" being non-malicious:
 https://www.threatcrowd.org/vote.php?vote=1&value=good.com

This will place a vote for "bad.com" being malicious:
 https://www.threatcrowd.org/vote.php?vote=0&value=bad.com

License
This data is available for free, and commercial use is allowed. It's licensed under http://www.apache.org/licenses/LICENSE-2.0
I make no guarantees to the quality of the data.
Posted by at

12 comments:

  1. Unknown21 March 2016 at 15:45

    Hi,

    I just added your IP Feed to FireHOL IP Lists, available at http://iplists.firehol.org/?ipset=threatcrowd

    ReplyDelete
  2. Chris21 March 2016 at 15:56

    Thanks! Just looked at firehol - it's a great idea and I'd love to see it cover more of the commercial providers to give a quick idea of quality.

    ReplyDelete
  3. Unknown21 March 2016 at 16:30

    Me too! Let's see if the commercial ones are willing to be compared...

    ReplyDelete
    Replies
    1. Chris21 March 2016 at 16:33

      True! Looks like someone compared a couple a while ago -> https://youtu.be/kstOKWL_OEk?t=18m12s

      Delete
    2. Unknown21 March 2016 at 16:45

      Interesting! This is what I found too: too few overlaps! It seems like the world is too big for anyone to cover the whole of it alone! My research shows that there are a few very interesting overlaps though: malware and abuse lists overlap to a great degree with proxies and anonymizers for example.

      Delete
  4. Erkan Şen2 August 2016 at 23:55

    Hi,

    Your site SSL settings gives an error and can't load. Could you please check it?

    "ERR_SSL_VERSION_OR_CIPHER_MISMATCH"

    ReplyDelete
  5. Erkan Şen2 August 2016 at 23:56

    Hi,

    Your site SSL settings gives an error and can't load. Could you please check it?

    "ERR_SSL_VERSION_OR_CIPHER_MISMATCH"

    ReplyDelete
  6. netguy6 November 2016 at 18:01

    Two problems with the domains.txt starting around Nov1..
    listing "fi" - what is that, it's not a domain
    a few listings "*.exe" - those are not domains

    ReplyDelete
  7. Chris26 November 2016 at 12:09

    Thanks! Sorry for the late reply. I've improved the validation.
    Still it's a bit hacky

    ReplyDelete
  8. Raquel11 January 2017 at 08:08

    How often are the list updated? It says every hour but there appears to be no changes.

    ReplyDelete
    Replies
    1. Unknown11 January 2017 at 08:12

      You can find its statistics here: http://iplists.firehol.org/?ipset=threatcrowd

      last time was updated Nov 16th 2016.

      Delete
    2. Raquel11 January 2017 at 08:20

      Thank you!

      Delete

Subscribe to: Post Comments (Atom)