Commits

Branch selector

 main

User selector

All users

All time

Commit History

Commits on May 11, 2024

• update readme

  

  whaleloops

  committed
  f43c693

  Copy full SHA for f43c693

Commits on May 9, 2024

• Copal task (#1803)

  Show description for 1980a13
  Erland366

  authored
  1980a13

  Copy full SHA for 1980a13

Commits on May 8, 2024

• Update flag `--hf_hub_log_args` in interface documentation (#1806)

  Show description for d32ce5c
  sepiatone

  authored
  d32ce5c

  Copy full SHA for d32ce5c

• add task for mmlu evaluation in arc multiple choice format (#1745)

  Show description for 9097ad3
  

  3 peopleauthored
  9097ad3

  Copy full SHA for 9097ad3

Commits on May 7, 2024

• Initial integration of the Unitxt to LM eval harness (#1615)

  Show description for 885f48d
  

  yoavkatz

  and

  haileyschoelkopf

  authored
  885f48d

  Copy full SHA for 885f48d

• Logging Updates (Alphabetize table printouts, fix eval tracker bug) (#1774) (#1791)

  Show description for d4a913c
  haileyschoelkopf

  authored
  d4a913c

  Copy full SHA for d4a913c

• Re-add Hendrycks MATH (no sympy checking, no Minerva hardcoded prompt) variant (#1793)

  Show description for d42a3e4
  haileyschoelkopf

  authored
  d42a3e4

  Copy full SHA for d42a3e4

• link to the example output on the hub (#1798)

  KonradSzafer

  authored
  20be169

  Copy full SHA for 20be169

• Fix Caching Tests ; Remove `pretrained=gpt2` default (#1775)

  haileyschoelkopf

  authored
  7fe2b93

  Copy full SHA for 7fe2b93

Commits on May 6, 2024

• Update `--tasks list` option in interface documentation (#1792)

  sepiatone

  authored
  66cf07e

  Copy full SHA for 66cf07e

• Provide ability for custom sampler for ConfigurableTask (#1616)

  Show description for ae72ceb
  LSinev

  authored
  ae72ceb

  Copy full SHA for ae72ceb

Commits on May 5, 2024

• Fix bug in setting until kwarg in openai completions (#1784)

  ciaranby

  authored
  30c060d

  Copy full SHA for 30c060d

• Fix README: change`----hf_hub_log_args` to `--hf_hub_log_args` (#1776)

  Show description for 297966f
  MuhammadBinUsman03

  authored
  297966f

  Copy full SHA for 297966f

• remove echo parameter in OpenAI completions API (#1779)

  Show description for c34986d
  djstrong

  authored
  c34986d

  Copy full SHA for c34986d

• limit fix (#1785)

  KonradSzafer

  authored
  cee785e

  Copy full SHA for cee785e

Commits on May 3, 2024

• eval tracker args fix (#1777)

  KonradSzafer

  authored
  18f4eb5

  Copy full SHA for 18f4eb5

• evaluation tracker implementation (#1766)

  Show description for 59cf408
  KonradSzafer

  authored
  59cf408

  Copy full SHA for 59cf408

Commits on May 2, 2024

• Add option to set OpenVINO config (#1730)

  Show description for e639471
  helena-intel

  authored
  e639471

  Copy full SHA for e639471

• vllm lora support (#1756)

  Show description for 83fd78a
  bcicc

  authored
  83fd78a

  Copy full SHA for 83fd78a

Commits on May 1, 2024

• upload new tasks (#1728)

  Show description for caaf9ab
  

  simran-arora

  and

  haileyschoelkopf

  authored
  caaf9ab

  Copy full SHA for caaf9ab

• Fix m_arc choices (#1760)

  Show description for f27c405
  

  jordane95

  and

  haileyschoelkopf

  authored
  f27c405

  Copy full SHA for f27c405

• Pile 10k new task (#1758)

  Show description for b898bda
  mukobi

  authored
  b898bda

  Copy full SHA for b898bda

• remove duplicated `num_fewshot: 0` (#1769)

  chujiezheng

  authored
  552eeae

  Copy full SHA for 552eeae

Commits on Apr 26, 2024

• Add filter registry decorator (#1750)

  Show description for f64e72f
  nikitml

  authored
  f64e72f

  Copy full SHA for f64e72f

• Support individual scrolls datasets (#1740)

  Show description for 9b49556
  giorgossideris

  authored
  9b49556

  Copy full SHA for 9b49556

Commits on Apr 25, 2024

• Fix Parameter Propagation for Tasks that have `include` (#1749)

  Show description for 0bafcef
  lintangsutawika

  authored
  0bafcef

  Copy full SHA for 0bafcef

• Add XNLIeu: a dataset for cross-lingual NLI in Basque (#1694)

  Show description for 594015b
  juletx

  authored
  594015b

  Copy full SHA for 594015b

• reference `--tasks list` in README (#1726)

  Show description for 80a056b
  nairbv

  authored
  80a056b

  Copy full SHA for 80a056b

Commits on Apr 18, 2024

• Adding retries and rate limit to toxicity tasks (#1620)

  sator-labs

  authored
  3196e90

  Copy full SHA for 3196e90

• fix error when appending eot_token_id for generate_until tasks (#1699)

  sergiopperez

  authored
  dc5eba8

  Copy full SHA for dc5eba8

Commits on Apr 16, 2024

• Add `neuralmagic` models for `sparseml` and `deepsparse` (#1674)

  Show description for 8b326be
  

  mgoin

  and

  haileyschoelkopf

  authored
  8b326be

  Copy full SHA for 8b326be

• Add delta weights model loading (#1712)

  Show description for 12a165d
  KonradSzafer

  authored
  12a165d

  Copy full SHA for 12a165d

Commits on Apr 8, 2024

• Update README.md (#1680)

  haileyschoelkopf

  authored
  7852985

  Copy full SHA for 7852985

Commits on Apr 7, 2024

• correction bug EleutherAI#1664 (#1670)

  Show description for e9a4054
  

  nicho2

  and

  haileyschoelkopf

  authored
  e9a4054

  Copy full SHA for e9a4054

Commits on Apr 5, 2024

• Anthropic Chat API (#1594)

  Show description for 27924d7
  

  tryumanshow

  and

  haileyschoelkopf

  authored
  27924d7

  Copy full SHA for 27924d7