Cache CI downloads #599

Activity

1. jbergstroem commented on Jan 25, 2017

   

   jbergstroem

   on Jan 25, 2017

   Member

   More actions

   @gibfahn you mean like setting up a squid/proxy/similar? In order for it to be effective I'm thinking we would want to do that per-continent. The arm setup at @rvagg's seems like a good first setup; second could be at DO us east?
2. gibfahn commented on Jan 25, 2017

   

   gibfahn

   on Jan 25, 2017

   MemberAuthor

   More actions

   cc/ @mhdawson
3. bnoordhuis commented on Jan 25, 2017

   

   bnoordhuis

   on Jan 25, 2017

   Member

   More actions

   @jbergstroem In the case of openssl-fips-2.0.9.tar.gz, it could be hosted on local disk or on the local network. That tarball was released 2 years ago and hasn't been the current one for 1.5 years.
4. jbergstroem commented on Jan 26, 2017

   

   jbergstroem

   on Jan 26, 2017

   Member

   More actions

   @bnoordhuis ok; similar to how we treat ICU I guess.
5. mhdawson commented on Jan 26, 2017

   

   mhdawson

   on Jan 26, 2017

   Member

   More actions

   The prior discusion/agreement was that we could host them on the main ci machine, we just never made progress on doing that.
6. bnoordhuis commented on May 26, 2017

   

   bnoordhuis

   on May 26, 2017

   Member

   More actions

   Backref to nodejs/node#13219 which was about the fips tarball download failing.
7. joaocgreis commented on Aug 15, 2017

   

   joaocgreis

   on Aug 15, 2017

   Member

   More actions

   While there's no better solution, this is what the ChakraCore Linux jobs use now:

   Bash function

   function cache_download {
     if [ -z "$1" ] || [ -z "$2" ] || [ -z "$3" ]; then
       echo "Missing argument"
       return 1
     fi
     URL=$1
     EXPECTED_SHA=$2
     DEST=$3
   
     mkdir -p "$HOME/cache_files"
     CACHE_FILE="$HOME/cache_files/$EXPECTED_SHA"
   
     CACHE_SHA="X"
     if [ -f "$CACHE_FILE" ]; then
       CACHE_SHA=`shasum -a 256 "$CACHE_FILE" | cut '-d ' -f1`
     fi
   
     if [ "X${CACHE_SHA}" != "X${EXPECTED_SHA}" ]; then
       rm "$CACHE_FILE" || true
       curl -L "$URL" -o "$CACHE_FILE"
       DOWNLOADED_SHA=`shasum -a 256 "$CACHE_FILE" | cut '-d ' -f1`
       if [ "X${DOWNLOADED_SHA}" != "X${EXPECTED_SHA}" ]; then
         echo Downloaded SHA256 mismatch.
         rm -v "$CACHE_FILE"
         return 1
       fi
     fi
   
     cp -fv "$CACHE_FILE" "$DEST"
   }

   
   
8. Trott commented on Mar 22, 2019

   

   Trott

   on Mar 22, 2019

   Member

   More actions

   Closing due to long period of inactivity. Feel free to re-open if this is a thing. I'm just trying to close stuff that has been ignored for sufficiently long that it seems likely it's not something we're going to get to.
9. Trott

   closed this as completedon Mar 22, 2019