Repository navigation
Set log level #3
Description
System information
- TensorFlow version (use command below): 1.15
Describe the current behavior
When using the current Java bindings for TensorFlow, the log gets filled with output from every operation when a model is evaluated. I was unable to find a way to set the log level via the Java API. Is this at all possible? If not, could you please consider adding this functionality in future versions?
Activity
karllessard commentedon Nov 4, 2019
@lucaro, can you please provide an example of such log? I don't see any in my case and I'm wondering if that is because your graph is taking a different path than mine.
lucaro commentedon Nov 4, 2019
Sure, the following is just a small snippet from the produced output. The lines which start with date and time information are written to the standard error stream while the others are written to standard out. The Graph used while generating this output is this one: https://github.com/lucaro/VideoSaliencyFilter/blob/master/omcnn2clstm.pb
karllessard commentedon Nov 5, 2019
Ok, it looks to me that you are logging device placement. This is normally controlled in Java by setting the
log_device_placementfield of the config proto to true and passing this proto in theSessionconstructor.I would have expect the default value to be false, are you passing such proto at the construction of your sessions?
lucaro commentedon Nov 5, 2019
Oops, the logDevicePlacement field was indeed set to true, must have accidentally autocompleted the wrong statement. After removing that statement, most, but not all of the log output disappears. The remiander looks like this:
The warnings look like they probably warrant their own issue at another time, but the rest looks like regular operation log, the level of which should be configurable somehow, right?
karllessard commentedon Nov 5, 2019
That I personally don't know, never tried to do it before. The messages you are showing are at the
INFOlevel and not debug traces, meaning that the developer was expecting them to be shown.I'll investigate a bit more how to turn those off but just to validate one last thing, are you in Android or just using the standard JDK?
Craigacp commentedon Nov 5, 2019
Looks like the protobuf warnings should go away if we use the latest protobuf release (see protocolbuffers/protobuf#3781).
lucaro commentedon Nov 5, 2019
Thanks! Regarding my environment, I'm currently using OpenJDK 12.
karllessard commentedon Nov 20, 2019
Hi @lucaro , sorry I didn’t came back sooner on this. After clearing device placement logs and upgrading protobuf version, is your execution still too verbose?
lucaro commentedon Nov 20, 2019
The log output shown above (#3 (comment)) is after the removal of the device placement log flag. I'm using the most recent available tensorflow version mavencentral offers, which is 1.15. I did not explicitly add protobuf to the project dependencies, the full list of dependencies is this: https://github.com/lucaro/VideoSaliencyFilter/blob/master/build.gradle
Do you think adding a protobuf dependency would help in the matter and if so, which one would that be?
karllessard commentedon Dec 2, 2019
Hi @lucaro , I'm really sorry for the late reply, I wanted to come back on this issue as soon as I find some time, which only happens now...
Digging in TF source code, I found out that there is an environment variable that set the minimum log level in the C++ core: https://github.com/tensorflow/tensorflow/blob/b82ab06d4b596a4aa964b2c78de3478e0cef6dd9/tensorflow/core/platform/default/logging.cc#L153
Can you set that environment variable
TF_CPP_MIN_LOG_LEVELto something higher than 0 and try again?lucaro commentedon Dec 2, 2019
I'm not entirely sure what causes it, but the amount of log output is different when I run the application from the IDE or from a jar. With the jar, the only remaining output is as follows, the environment variable you mentioned does not appear to make any difference to it.
karllessard commentedon Dec 2, 2019
OK so from the JAR, this kind of log seems normal to me (the
protobufwarning might be avoided by overloading the version of their artifact, like @Craigacp mentioned, but it is not something we need to worry about).Which IDE do you use? Are you sure the environment variable I've mentioned is not set somewhere in the execution settings of your app?
lucaro commentedon Dec 3, 2019
It's not only the protobuf warning, the last line comes from TensorFlow.
I'm using IntelliJ and did not set any other parameters for execution.
karllessard commentedon Dec 3, 2019
The line warning you that TF is not optimized to support your CPU instruction set has always been there so I don't see it as an issue.
lucaro commentedon Dec 4, 2019
Well, at this point it's more of a philosophical question I guess but shouldn't the log output of a library, especially the one concerned with information rather than errors, be controllable by the main application using the library?
3 remaining items
karllessard commentedon Dec 5, 2019
Thanks @lucaro,
do you still see "undesirable" messages in the logs after setting this? (i.e. anything that is just informative?)
lucaro commentedon Dec 6, 2019
No, after setting this, only the protobuf warning remains. Since there is no way to set this from within the program though, this is only a solution for this particular instance and not for the log management in general. That would however rather be a feature request towards the C++ part of TensorFlow, right?
karllessard commentedon Dec 7, 2019
Yes, I think we need to add an endpoint to set the value of this variable in a friendly matter, we’ll take a look at it.
For the protobuf warning, I’ll come back to you as soon as I know for sure it is a C++ runtime issue, which is my feeling right now
Craigacp commentedon Dec 7, 2019
The IllegalReflectiveAccess warning is because the protobuf jar is messing with fields it shouldn't be. Its emitted on Java 9 and newer. Updating to a newer version will probably fix it.
karllessard commentedon Dec 8, 2019
Oh, of course the warning to come from a Java artifact since it is about an illegal reflective access... But I can't reproduce it on my side, even if I include an additional dependency to
org.tensorflow:protoand I'm using JDK 11.@lucaro , are you sure that this warning does not come from another artifact that your application might depend on?
lucaro commentedon Dec 9, 2019
TensorFlow is the only library I'm using in this particular project which has a dependency to protobuf. In case it helps, this is the complete list of dependencies: https://github.com/lucaro/VideoSaliencyFilter/blob/master/build.gradle
Craigacp commentedon Dec 10, 2019
Looks like it's fixed by using protobuf-java 3.7.0 or newer (https://github.com/protocolbuffers/protobuf/releases/tag/v3.7.0). The 1.15.0 proto artifact depends on protobuf-java 3.5.1. @karllessard, @sjamesr, where does the org.tensorflow:proto artifact come from? Is it one of the things we'll produce here, or is it produced by Google?
sjamesr commentedon Dec 10, 2019
Google has been doing the releases up to this point, I don't know exactly what the plan is for this artifact in the future.
karllessard commentedon Jan 23, 2020
An new issue has been created for protobuf support in new TF Java: #21
Merge pull request tensorflow#3 from tensorflow/master
silvanheller commentedon Mar 3, 2022
Bumping this issue, is there any way to set the log level within the Java Code now that the corresponding issue has been resolved?
Craigacp commentedon Mar 3, 2022
We spent some time looking into exporting TF logging messages into Java so they can be logged using a Java logging framework, but unfortunately that effort stalled as the TF log handler doesn't behave correctly - tensorflow/tensorflow#44995, #345.
Sparse tensor (#3)
Migrate ndarray (#634)
Sparse tensor (tensorflow#3)