Friday, July 8, 2016

Benchmark a simple no-op Kafka consumer using Kafka client Java library

Test environment is as follows:

```
CPU: Intel L5640 2.26 GHz 6 cores * 2 EA
Memory: SAMSUNG PC3-10600R 4 GB * 4 EA
HDD: TOSHIBA SAS 10,000 RPM 300 GB * 6 EA

OS: CentOS release 6.6 (Final)

Kafka server 0.9.0.0
Kafka client Java library 0.9.0.1
```

I used a custom tool as follows:

```
git clone https://github.com/izeye/kafka-consumer.git
cd kafka-consumer/
./gradlew clean bootRepackage
java -jar build/libs/kafka-consumer-1.0.jar --spring.profiles.active=noop --kafka.consumer.bootstrap-servers=1.2.3.4:9092 --kafka.consumer.group-id=logstash --kafka.consumer.topic=some-log
```

I got the following result:

```
# of consumed logs per second: 29531
# of consumed logs per second: 38848
# of consumed logs per second: 28747
# of consumed logs per second: 49191
# of consumed logs per second: 28797
```

It consumed from 30K to 50K.

No comments:

Post a Comment