Skip to content

Fix memory leak: reuse loggers instead of registering one per call - #114

Open
vcabieles wants to merge 1 commit into
AuthorizeNet:masterfrom
vcabieles:fix/logger-memory-leak
Open

vcabieles wants to merge 1 commit into
AuthorizeNet:masterfrom
vcabieles:fix/logger-memory-leak

Conversation

@vcabieles

Copy link
Copy Markdown

Fixes #113. Also fixes #97.

Problem

Since 1.0.9, getLogger() in lib/logger.js appends a random number to the logger category before calling winston.loggers.get(). winston's Container keeps every id for the life of the process, so every APIContracts.* / APIControllers.* object registers a new logger that is never reused or freed. This happens even when logging is disabled, which is the default.

Each request/controller pair keeps 5 loggers and about 57KB of heap. Long-running servers grow until they run out of heap. In our production payment service this crashed Node 6 times in one day.

Fix

Key the logger on the category plus its logger settings (enabled, level, location) instead of a random number:

  • The same category and settings reuse one logger.
  • Different settings still get their own logger. winston.loggers.get() ignores options for an existing id, which I assume is why the random suffix was added.
  • If the logger already exists, it's returned early, so file transports aren't rebuilt on every call when logging is enabled.

Log labels now show the plain category name (e.g. ApiOperationBase) instead of ApiOperationBase483920117.

Results

Request + controller pairs (GetTransactionDetails), no network, yielding to the event loop between batches, heap measured after GC (Node v25.2.1):

1,000 calls 5,000 calls 10,000 calls
1.0.10 67.3 MB, 5,000 loggers 294.0 MB, 25,000 loggers 577.6 MB, 50,000 loggers
1.0.10 + this PR 10.5 MB, 4 loggers 10.8 MB, 4 loggers 10.7 MB, 4 loggers
1.0.8 (for reference) 11.0 MB 11.0 MB 11.0 MB

Tests

New test/test-logger.js. It runs offline with no sandbox credentials:

  • 1,000 request/controller pairs register no new loggers
  • The same category and settings return the same logger
  • Different logger settings return separate loggers with the right level

npx mocha test/test-logger.js: 3 passing with this change. Without it, 2 tests fail. The new test file passes eslint. The 2 eslint errors in lib/logger.js (semi, no-prototype-builtins) were already on master, and I left them alone to keep the diff small.

getLogger() appended a random number to the category before calling
winston.loggers.get(). winston's container keeps every id for the life of
the process, so every APIContracts/APIControllers object registered a new
logger that was never freed (~5 loggers and ~57KB of heap per
request/controller pair). Long-running servers grew until they ran out of
heap. Fixes AuthorizeNet#113 and AuthorizeNet#97.

Key the logger on the category plus its logger settings (enabled, level,
location) instead. The same settings reuse one logger, and different
settings, such as an externalConfig, still get their own, since
winston.loggers.get() ignores options for an existing id. Return the
existing logger early so file transports are not rebuilt on every call
when logging is enabled.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Memory leak: getLogger() registers a new winston logger for every SDK object (since 1.0.9) Possible memory leak with latest release 1.0.9

1 participant