Repositories list

• system-dev-ops

  Public
  devops tools

  Shell

  •

  Apache License 2.0

  •0•0•0•0•Updated Jul 26, 2024Jul 26, 2024

• sous-chef

  Public
  Configurable Data Analytics Pipeline

  sous-chef

  Python

  •0•1•8•0•Updated Jul 25, 2024Jul 25, 2024

• news-search-api

  Public
  Internal API server that offers search access to the Media Cloud Online News Archive (in Elasticsearch).

  Python

  •

  GNU Affero General Public License v3.0

  •3•1•6•1•Updated Jul 24, 2024Jul 24, 2024

• mc-providers

  Public
  Internal library to allow querying multiple media platforms with a consistent API.

  Python

  •1•0•3•0•Updated Jul 24, 2024Jul 24, 2024

• story-indexer

  Public
  The core pipeline used to ingest online news stories in the Media Cloud archive.

  Python

  •

  Apache License 2.0

  •4•1•31•2•Updated Jul 24, 2024Jul 24, 2024

• web-search

  Public
  Code that drives the public web-based tools for the Media Cloud Online News Archive and Directory.

  social-mediaresearch-toolsocial-science+ 1media-analytics

  JavaScript

  •

  Apache License 2.0

  •12•9•36•1•Updated Jul 23, 2024Jul 23, 2024

• sc-buffet

  Public
  Sous-chef buffet - Self-service data access for sous-chef.

  Python

  •0•0•5•0•Updated Jul 22, 2024Jul 22, 2024

• system-metrics

  Public
  Daily performance metrics for the mediacloud application

  sous-chef

  Python

  •0•0•1•0•Updated Jul 5, 2024Jul 5, 2024

• mc-manage

  Public
  Python

  •0•0•0•0•Updated Jun 25, 2024Jun 25, 2024

• mediacloud-news-client

  Public
  An internal client library to access the new Mediacloud news archive search.

  Python

  •

  Apache License 2.0

  •2•0•3•1•Updated Jun 14, 2024Jun 14, 2024

• rss-fetcher

  Public
  Intelligently fetch lists of URLs from a large collection of RSS Feeds as part of the Media Cloud Directory.

  Python

  •

  Apache License 2.0

  •5•5•14•1•Updated Jun 2, 2024Jun 2, 2024

• cliff-api-client

  Public
  A Python client for the CLIFF geoparsing tool

  Python

  •

  MIT License

  •5•5•0•1•Updated May 21, 2024May 21, 2024

• api-client

  Public
  Public client for consuming content from the Media Cloud Online News Archive & Directory.

  Python

  •

  Apache License 2.0

  •24•68•1•1•Updated May 7, 2024May 7, 2024

• metadata-lib

  Public
  How Media Cloud approaches extracting metadata from online news stories

  Python

  •

  Apache License 2.0

  •3•11•3•0•Updated May 7, 2024May 7, 2024

• wayback-news-client

  Public
  A client library to access the Wayback Machine news archive search.

  Python

  •

  Apache License 2.0

  •2•4•1•0•Updated Dec 15, 2023Dec 15, 2023

• web-tools

  Public archive
  The shared repository for Media Cloud web apps (Explorer, Source Manager, Topic Mapper)

  flaskreactjscivic-tech+ 3media-analyticspythonmedia-cloud

  JavaScript

  •

  Apache License 2.0

  •30•63•31•4•Updated Dec 14, 2023Dec 14, 2023

• api-tutorial-notebooks

  Public
  A set of jupyter notebooks demonstrating how to use the Media Cloud API.

  Jupyter Notebook

  •14•33•0•0•Updated Dec 13, 2023Dec 13, 2023

• backend

  Public archive
  Media Cloud is an open source, open data platform that allows researchers to answer quantitative questions about the content of online media.

  medianewspapercontent-analysis+ 3news-medianewsmediaonline-news

  Python

  •

  GNU Affero General Public License v3.0

  •87•277•131•25•Updated Nov 20, 2023Nov 20, 2023

• a-catchall-page

  Public
  Dokku app that serves a static HTML catch-all page, displayed for bad domains

  HTML

  •0•0•0•0•Updated Oct 25, 2023Oct 25, 2023

• cliff-homepage

  Public
  A simple homepage for the CLIFF project

  HTML

  •

  MIT License

  •1•1•0•0•Updated May 30, 2023May 30, 2023

• ultimate-sitemap-parser

  Public
  Ultimate Website Sitemap Parser

  pythonsitemappython3+ 5robots-txtpython-3xml-sitemapsitemap-xmlxml-sitemap-parser

  Python

  •

  Other

  •64•174•18•4•Updated May 17, 2023May 17, 2023

• nyt-news-labeler

  Public
  Tag news stories based on models trained on the NYT corpus.

  Python

  •

  Apache License 2.0

  •12•39•1•6•Updated Mar 1, 2023Mar 1, 2023

• feed_seeker

  Public
  Find rss, atom, xml, and rdf feeds on webpages

  Python

  •

  MIT License

  •12•31•4•1•Updated Feb 27, 2023Feb 27, 2023

• sentence-splitter

  Public
  Text to sentence splitter using heuristic algorithm by Philipp Koehn and Josh Schroeder.

  nlptokenizersentence-splitting

  Python

  •

  Other

  •30•223•2•2•Updated Nov 7, 2022Nov 7, 2022

• glimpse

  Public archive
  Get a glimpse of attention to a topic on social media.

  Python

  •

  Apache License 2.0

  •2•2•8•0•Updated Sep 19, 2022Sep 19, 2022

• word-embeddings-server

  Public archive
  Helpful micro-service to return results from word2vec models

  Python

  •

  MIT License

  •4•2•0•0•Updated Jul 29, 2022Jul 29, 2022

• cliff-annotator

  Public archive
  A lightweight server to allow HTTP requests to the Stanford Named Entity Recognized and a heavily modified CLAVIN geoparser.

  Java

  •

  Apache License 2.0

  •34•119•9•10•Updated May 20, 2022May 20, 2022

• backup-collection-maker

  Public archive
  Notebook demonstrating how to create and update a Media Cloud collection.

  Jupyter Notebook

  •0•0•0•0•Updated Mar 30, 2022Mar 30, 2022

• backend-temporal-server-config

  Public archive
  Temporal server configuration

  0•0•0•0•Updated Jan 4, 2022Jan 4, 2022

• postgresql-citus-aws-graviton2

  Public archive
  PostgreSQL built for AWS Graviton2

  awspostgresqlgraviton2+ 1aws-graviton2

  1•2•0•0•Updated Dec 30, 2021Dec 30, 2021