[Integration] Decimal test files are huge

[pitrou]

Describe the bug, including details regarding any error messages, version, and platform.

For some reason, the decimal files generated by the integration test suite have as many record batches as columns.
This looks like a programming error rather than something deliberate:

arrow/dev/archery/archery/integration/datagen.py

Lines 1516 to 1540 in 3e0ca5b

	def generate_decimal128_case():
	fields = [
	DecimalField(name='f{}'.format(i), precision=precision, scale=2,
	bit_width=128)
	for i, precision in enumerate(range(3, 39))
	]
	possible_batch_sizes = 7, 10
	batch_sizes = [possible_batch_sizes[i % 2] for i in range(len(fields))]
	# 'decimal' is the original name for the test, and it must match
	# provide "gold" files that test backwards compatibility, so they
	# can be appropriately skipped.
	return _generate_file('decimal', fields, batch_sizes)
	def generate_decimal256_case():
	fields = [
	DecimalField(name='f{}'.format(i), precision=precision, scale=5,
	bit_width=256)
	for i, precision in enumerate(range(37, 70))
	]
	possible_batch_sizes = 7, 10
	batch_sizes = [possible_batch_sizes[i % 2] for i in range(len(fields))]
	return _generate_file('decimal256', fields, batch_sizes)

See in particular:

arrow/dev/archery/archery/integration/datagen.py

Line 1524 in 3e0ca5b

batch_sizes = [possible_batch_sizes[i % 2] for i in range(len(fields))]

Component(s)

Archery, Integration