Skip to content

json-read: JSON readtable error with python 3.6.3 #135

Open

Description

@lengyueyang

We I use python 3.6.2 and 3.6.2, when execute, occur json-read: JSON readtable error, but when use python 3.5.2, all is ok

Activity

  1. dcherian commented on Nov 13, 2017

    @dcherian

    I had this problem and had to downgrade.

    Can we have a section in README that mentions the latest version of python this has been tested on?

  2. andrewsanchez commented on Nov 13, 2017

    @andrewsanchez

    Neither downgrading with conda install python=3.5 (resulted in 3.5.4) nor conda install python=3.5.2 worked for me.

  3. dcherian commented on Nov 13, 2017

    @dcherian

    Oh I meant downgrade ob-ipython. I haven't tried with python 3.5

  4. dcherian commented on Nov 17, 2017

    @dcherian

    Can someone who's using the latest ob-ipython please comment with versions of python / ipython / jupyter they're using?

  5. nicholas-miklaucic commented on Nov 29, 2017

    @nicholas-miklaucic

    I think I figured this out, at least on my system (latest version of ob-ipython, ipython version 6.1.0, python version 3.6.2 with Anaconda). I got this error too, and I fixed it by using exec-path-from-shell to set the PATH correctly. Install that from MELPA and put this in your init file before loading ob-ipython:

    (when (memq window-system '(mac ns x))
      (exec-path-from-shell-initialize)) 
    

    I think the reason this happens is because it uses jupyter to get a JSON output of the kernels, but it runs that via shell-command, which doesn't use the exec-path variable the rest of Emacs uses. You should be good if M-! jupyter -h returns a normal help entry and not zsh:1:command not found: 'jupyter' or equivalent.

    This is probably a documentation problem moreso than a code problem, as I'm not sure how you're supposed to be able to divine where the jupyter executable is in elisp.

  6. dcherian commented on Dec 3, 2017

    @dcherian

    Hmm... I've always had exec-path-from-shell configured and running. M-! jupyter -h returns a normal help entry too.

    Is there a way to see the string json-read is having trouble decoding? My elisp isn't good enough to figure it out.

  7. nicholas-miklaucic commented on Dec 3, 2017

    @nicholas-miklaucic

    We must have different errors with the same error message. Can you try invoking toggle-debug-on-errorand then running it, or if that doesn't work trying toggle-debug-on-entry with json-read?

  8. dcherian commented on Dec 4, 2017

    @dcherian

    @nicholas-miklaucic Thanks!

    Using debug-on-entry showed this (newlines added for clarity):

      json-read-from-string("/home/deepak/.bashrc: line 115: bind: warning: line editing not enabled\n
    {\n 
     \"kernelspecs\": {\n    \"python3\": {\n     
     \"resource_dir\": \"/home/deepak/anaconda3/share/jupyter/kernels/python3\",\n      \"spec\": {\n        \"argv\": [\n         
     \"/home/deepak/anaconda3/bin/python\",\n          \"-m\",\n          \"ipykernel_launcher\",\n          \"-f\",\n          \"{connection_file}\"\n        ],\n        \"env\": {},\n       
     \"display_name\": \"Python 3\",\n        \"language\": \"python\"\n      }\n    }\n  }\n}\n")
    

    Once I comment the line that makes bash throw a warning, things work!

    @gregsexton Maybe a regexp to select the bit that begins with {\n and ends with \n") before passing it to json-read would do the trick? Perhaps you could catch the error and display the string so it's clear what json-read is choking on?

  9. georgek commented on Dec 4, 2017

    @georgek

    This isn't working at all for me. Using Python 3.6.2 and getting the same JSON read table error. This is me trying to run the example:

    Debugger entered--Lisp error: (json-readtable-error)

      signal(json-readtable-error nil)
      json-read()
      (cons (json-read) acc)
      (setq acc (cons (json-read) acc))
      (while (not (= (point) (point-max))) (setq acc (cons (json-read) acc)) (forward-line))
      (let (acc) (while (not (= (point) (point-max))) (setq acc (cons (json-read) acc)) (forward-line)) (nreverse acc))
      (let ((json-array-type (quote list))) (let (acc) (while (not (= (point) (point-max))) (setq acc (cons (json-read) acc)) (forward-line)) (nreverse acc)))
      ob-ipython--collect-json()
      (if (> ret 0) (ob-ipython--dump-error (buffer-string)) (goto-char (point-min)) (ob-ipython--collect-json))
      (let ((ret (apply (quote call-process-region) code nil (ob-ipython--get-python) nil t nil (list "--" ob-ipython-client-path "--conn-file" name "--execute")))) (if (> ret 0) (ob-ipython--dump-error (buffer-string)) (goto-char (point-min)) (ob-ipython--collect-json)))
      (progn (let ((ret (apply (quote call-process-region) code nil (ob-ipython--get-python) nil t nil (list "--" ob-ipython-client-path "--conn-file" name "--execute")))) (if (> ret 0) (ob-ipython--dump-error (buffer-string)) (goto-char (point-min)) (ob-ipython--collect-json))))
      (unwind-protect (progn (let ((ret (apply (quote call-process-region) code nil (ob-ipython--get-python) nil t nil (list "--" ob-ipython-client-path "--conn-file" name "--execute")))) (if (> ret 0) (ob-ipython--dump-error (buffer-string)) (goto-char (point-min)) (ob-ipython--collect-json)))) (and (buffer-name temp-buffer) (kill-buffer temp-buffer)))
      (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (let ((ret (apply (quote call-process-region) code nil (ob-ipython--get-python) nil t nil (list "--" ob-ipython-client-path "--conn-file" name "--execute")))) (if (> ret 0) (ob-ipython--dump-error (buffer-string)) (goto-char (point-min)) (ob-ipython--collect-json)))) (and (buffer-name temp-buffer) (kill-buffer temp-buffer))))
      (let ((temp-buffer (generate-new-buffer " *temp*"))) (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (let ((ret (apply ... code nil ... nil t nil ...))) (if (> ret 0) (ob-ipython--dump-error (buffer-string)) (goto-char (point-min)) (ob-ipython--collect-json)))) (and (buffer-name temp-buffer) (kill-buffer temp-buffer)))))
      ob-ipython--execute-request("%matplotlib inline\nimport matplotlib.pyplot as plt\nimport numpy as np" "default")
      (ob-ipython--eval (ob-ipython--execute-request (org-babel-expand-body:generic (encode-coding-string body (quote utf-8)) params (org-babel-variable-assignments:python params)) (ob-ipython--normalize-session session)))
      (let ((ret (ob-ipython--eval (ob-ipython--execute-request (org-babel-expand-body:generic (encode-coding-string body (quote utf-8)) params (org-babel-variable-assignments:python params)) (ob-ipython--normalize-session session))))) (if ret (progn (ob-ipython--process-response ret file result-type))))
      (let* ((file (cdr (assoc :ipyfile params))) (session (cdr (assoc :session params))) (result-type (cdr (assoc :result-type params)))) (ob-ipython--create-kernel (ob-ipython--normalize-session session) (cdr (assoc :kernel params))) (let ((ret (ob-ipython--eval (ob-ipython--execute-request (org-babel-expand-body:generic (encode-coding-string body ...) params (org-babel-variable-assignments:python params)) (ob-ipython--normalize-session session))))) (if ret (progn (ob-ipython--process-response ret file result-type)))))
      ob-ipython--execute-sync("%matplotlib inline\nimport matplotlib.pyplot as plt\nimport numpy as np" ((:colname-names) (:rowname-names) (:result-params "replace" "drawer") (:result-type . value) (:results . "replace drawer") (:exports . "code") (:cache . "no") (:noweb . "no") (:hlines . "no") (:tangle . "no") (:session)))
      (if (cdr (assoc :async params)) (ob-ipython--execute-async body params) (ob-ipython--execute-sync body params))
      org-babel-execute:ipython("%matplotlib inline\nimport matplotlib.pyplot as plt\nimport numpy as np" ((:colname-names) (:rowname-names) (:result-params "replace" "drawer") (:result-type . value) (:results . "replace drawer") (:exports . "code") (:cache . "no") (:noweb . "no") (:hlines . "no") (:tangle . "no") (:session)))
      org-babel-execute-src-block(nil ("ipython" "%matplotlib inline\nimport matplotlib.pyplot as plt\nimport numpy as np" ((:colname-names) (:rowname-names) (:result-params "drawer" "replace") (:result-type . value) (:results . "drawer replace") (:exports . "code") (:session) (:tangle . "no") (:hlines . "no") (:noweb . "no") (:cache . "no")) "" nil 714 "(ref:%s)"))
      org-ctrl-c-ctrl-c(nil)
      funcall-interactively(org-ctrl-c-ctrl-c nil)
      call-interactively(org-ctrl-c-ctrl-c nil nil)
      command-execute(org-ctrl-c-ctrl-c)
    
    

    There are also seem to be problems with it killing the jupyter process. I have to kill it manually otherwise I get errors from ZMQ saying the port is already in use.

  10. nicholas-miklaucic commented on Dec 4, 2017

    @nicholas-miklaucic

    Try invoking toggle-debug-on-entry with json-read to see what it's choking on: your error seems different as it's for running code, AFAIK.

  11. dcherian commented on Dec 4, 2017

    @dcherian

    You'll also want to make sure that jupyter isn't writing any warnings also. I had to remove those too.

  12. georgek commented on Dec 5, 2017

    @georgek

    I get a buffer called *ob-ipython-kernel-default* with some output from starting a jupyter console. It seems that json-read is being run on this buffer which doesn't contain anything that looks like JSON:

    Jupyter console 5.2.0
    
    Python 3.6.2 (default, Sep 28 2017, 13:58:54) 
    Type 'copyright', 'credits' or 'license' for more information
    IPython 6.2.0 -- An enhanced Interactive Python. Type '?' for help.
    
    
    
    In [1]: 
    
  13. dcherian commented on Dec 5, 2017

    @dcherian

    @georgek That's how it looks like when it works.

    Set debug-on-entry to stop at ob-ipython--collect-json. When it does hit c to continue, do this twice and it should show the string it's trying to read. That'll tell you what it's choking on

    EDIT: Not sure that will work actually but you can try. I found the second answer here helpful: https://stackoverflow.com/questions/3257480/how-to-debug-elisp#3258205

  14. georgek commented on Dec 5, 2017

    @georgek

    I've tried using edebug and stepping through. The difficulty seems to be that in ob-ipython--execute-request it runs ob-ipython--collect-json in a temporary buffer. I can't figure out how to look at what's being written in the temp buffer, though.

  15. georgek commented on Dec 5, 2017

    @georgek

    OK, I finally figured it out. It's not easy to debug something that uses with-temp-buffer, but eventually I figured out I could manually repalce that with a real buffer and see the output. Turns out my ipython prints an error about being run in a virtualenv which is not proper JSON. So now I need to figure out how to suppress that warning (which I think is safe).

    I succumbed to installing jupyter/ipython within each virtualenv and now there is no error and it works.

  16. 12 remaining items

  17. nanounanue commented on Dec 31, 2017

    @nanounanue

    @dcherian What do you mean? Could you be more specific?

  18. aayusuff commented on Dec 31, 2017

    @aayusuff

    You can set ob-ipython-command to the path of jupyter. e,g
    (setq ob-ipython-command "/home/name/miniconda3/bin/jupyter") . You can then restart emacs.

  19. nanounanue commented on Dec 31, 2017

    @nanounanue

    @aayusuff I am using pyenv-mode for that. ob-ipython is picking the right path. But I will try your suggestion anyway.

  20. theottm commented on Sep 23, 2018

    @theottm

    Hello everyone,

    when I use ob-ipython in Emacs launched from a shell with an active virtualenv (using pyenv) I get the same json-readtable error. My code runs on the kernel though, I could verify it by using the interactive session with C-c C-v C-z. However I don't get any output in the result block.

    Now when I launch Emacs from my basic system environment and then use pyen-mode-set to activate the exact same virtualenv from within Emacs, everything works fine.

    I am happy with that but still curious to understand what exactly is going wrong.
    I added @dangirsh 's fix dangirsh@df13b94 to the ob-iython--execute-request function to be able to have a look at the json ob-ipython and the kernel use to communicate and I finally found where the problem was in my case : the kernel throws a warning message that is not json formated.

    /home/teddd/.local/lib/python3.5/site-packages/IPython/core/interactiveshell.py:763: UserWarning: Attempting to work in a virtualenv. If you encounter problems, please install IPython inside the virtualenv. warn("Attempting to work in a virtualenv. If you encounter problems, please "

    Thus when reading this message, ob-ipython--collect-json returns an error number being the first letter encountered in the error message in decimal form. (In my case error 47 corresponds to the "/" character) This explains why this number is so random and cryptic.

    As I see it the problem is not only virtualenvs, but also the way ob-ipython handles outputs coming from the kernel. This non json message should clearly be redirected to the error/warning buffer. Maybe there should be a filter ? Is there already such a function somewhere else in the code ? How are error messages normaly handled @gregsexton ?

    Of course the simplest way would be to just run pyenv or get the PATH from the shell and not have any error message, but this wouldn't make the program more robust. Who knows if the next version of Jupyter doesn't throw wierd stuff on our poor json-read ?

  21. jchkoch commented on Sep 25, 2018

    @jchkoch

    Hello everyone,

    Currently, my ob-ipython setup returned to having similar error as discussed in this issue when I attempt to run a ipython code block from within a org mode file. If others think I should open a new issue, I will do so; however, after reading the comments under this issue I felt that my error was similar enough to warrant posting a comment here.

    Now to the actual error message I am seeing:

    error in process sentinel: json-read: JSON readtable errorxo error in process sentinel: JSON readtable error

    When toggling debug on error, I get the following backtrace:

    Debugger entered--Lisp error: (json-readtable-error)
      signal(json-readtable-error nil)
      json-read()
      (cons (json-read) acc)
      (setq acc (cons (json-read) acc))
      (while (not (= (point) (point-max))) (setq acc (cons (json-read) acc)) (forward-line))
      (let (acc) (while (not (= (point) (point-max))) (setq acc (cons (json-read) acc)) (forward-line)) (nreverse acc))
      (let ((json-array-type (quote list))) (let (acc) (while (not (= (point) (point-max))) (setq acc (cons (json-read) acc)) (forward-line)) (nreverse acc)))
      ob-ipython--collect-json()
      (ob-ipython--eval (ob-ipython--collect-json))
      (cons (ob-ipython--eval (ob-ipython--collect-json)) (symbol-value G9))
      (apply (symbol-value G10) (cons (ob-ipython--eval (ob-ipython--collect-json)) (symbol-value G9)))
      (save-current-buffer (set-buffer (process-buffer proc)) (goto-char (point-min)) (apply (symbol-value G10) (cons (ob-ipython--eval (ob-ipython--collect-json)) (symbol-value G9))))
      (progn (save-current-buffer (set-buffer (process-buffer proc)) (goto-char (point-min)) (apply (symbol-value G10) (cons (ob-ipython--eval (ob-ipython--collect-json)) (symbol-value G9)))) (ob-ipython--maybe-run-async))
      (if (not (process-live-p proc)) (progn (save-current-buffer (set-buffer (process-buffer proc)) (goto-char (point-min)) (apply (symbol-value G10) (cons (ob-ipython--eval (ob-ipython--collect-json)) (symbol-value G9)))) (ob-ipython--maybe-run-async)))
      (lambda (G9 G10 proc state) (if (not (process-live-p proc)) (progn (save-current-buffer (set-buffer (process-buffer proc)) (goto-char (point-min)) (apply (symbol-value G10) (cons (ob-ipython--eval (ob-ipython--collect-json)) (symbol-value G9)))) (ob-ipython--maybe-run-async))))(--args-- --callback-- #<process execute> "finished\n")
      apply((lambda (G9 G10 proc state) (if (not (process-live-p proc)) (progn (save-current-buffer (set-buffer (process-buffer proc)) (goto-char (point-min)) (apply (symbol-value G10) (cons (ob-ipython--eval (ob-ipython--collect-json)) (symbol-value G9)))) (ob-ipython--maybe-run-async)))) --args-- --callback-- (#<process execute> "finished\n"))
      (lambda (&rest --cl-rest--) (apply (quote (lambda (G9 G10 proc state) (if (not (process-live-p proc)) (progn (save-current-buffer (set-buffer ...) (goto-char ...) (apply ... ...)) (ob-ipython--maybe-run-async))))) (quote --args--) (quote --callback--) --cl-rest--))(#<process execute> "finished\n")
    

    Furthermore, the following is additional information regarding package versions in my current setup:

    jupyter-console --version 5.2.0
    jupyter --version 4.4.0
    ipython --version 6.4.0
    python --version Python 3.6.4

    emacs --version
    GNU Emacs 25.3.1
    Copyright (C) 2017 Free Software Foundation, Inc.
    GNU Emacs comes with ABSOLUTELY NO WARRANTY.
    You may redistribute copies of GNU Emacs
    under the terms of the GNU General Public License.
    For more information about these matters, see the file named COPYING.
    

    I have the most recent version of ob-ipython installed directly from the git repository and I am running a openSUSE Leap 15.0 linux machine.
    I have tried some of the solutions proposed in this issue to workaround this error but none of them worked in my case. The only thing that helped is to add (exec-path-from-shell-initialize) in my emacs configuration before loading ob-ipython which only caused no backtrace to be generated while still displaying the error json-read: JSON readtable error.

    If anyone has any ideas for how I can fix this issue I am experiencing, I would appreciate the help.

  22. theottm commented on Sep 26, 2018

    @theottm

    @jchkoch your error seem to be different, but the problem might be the same. The problem is that this error message doesn't tell a lot about where the problem reside. Generally though it is a problem of the way ob-ipython and your python environment interact.

    Have you checked if the env is well set in your org file ? You can try shell-command RET python --version RET and the others. If there is a problem here, using pyenv and its emacs plugin might be a solution.

    To address your problem more specifically, I advice you to use this hack dangirsh@df13b94 to be able to have a look at the json and understand what in it causes the reader to crash.

    I also had good results using edebug. It is implemented in Emacs and allows you to follow step by step the program in the elisp source code, knowing what data is coming in and out of each step. You could thus see more precisely where it crashes in the json-read function. But my guess is just that the json is badly formatted, have a look a it first :)

    Good luck !

  23. jchkoch commented on Sep 27, 2018

    @jchkoch

    The json-readtable error that I reported 2 days ago (#135 (comment)) disappeared when I restarted emacs today. So far, I have not ascertained why the issue disappeared. Anyway, thanks for the reply @theottm and I will keep this in mind if the error ever comes back.

  24. theottm commented on Sep 29, 2018

    @theottm

    glad to hear it !

  25. added a commit that references this issue on Dec 26, 2018
  26. ganesh-krishnan commented on Mar 2, 2019

    @ganesh-krishnan

    I was seeing the same error. #135 (comment) worked. Thank you!

  27. thcoffee-msft commented on Apr 1, 2019

    @thcoffee-msft

    I was getting a similar problem, but using edebug revealed that I was getting extra messages after the JSON response rather than before. I extended the solution in #135 (comment) to remove anything around or between top-level JSON objects in the response, which seems to be working for me:

    ; kill extraneous content in ipython returns between top-level JSON objects
    (advice-add 'ob-ipython--collect-json :before
                (lambda (&rest args)
                  (let ((start (point)))
                    (set-mark (point))
                    (while (re-search-forward "{" nil t)
                      (backward-char)
                      (kill-region (region-beginning) (region-end))
                      (re-search-forward "}\n" nil t)
                      (set-mark (point)))
                    (end-of-buffer)
                    (kill-region (region-beginning) (region-end))
                    (goto-char start))))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions