Install fay with cabal new-install

I tried to install fay and fay-base with cabal new-install today. I had to bump a few version constraints but both packages managed to install, however when I tried to run fay it could not find fay-base. Any pointers would be appreciated. Thanks.
Best wishes,
Henry Laxen

[swamp-agr]

Hi Henry,

Have you tried to determine which package-db was populated?

ghc-pkg describe fay-base

I would also recommend to use https://github.com/faylang/fay/wiki/Installing-and-running despite it's outstanding. HASKELL_PACKAGE_SANDBOX environment variable is a trick to make fay-base visible to fay.

Could you also please provide which ghc version you are using.

Thanks,
@swamp-agr

When I clone the git repository and try to build fay with cabal new-build:

()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()

cabal new-build all
Resolving dependencies...
cabal: Could not resolve dependencies:
[__0] trying: fay-0.24.0.1 (user goal)
[__1] next goal: base (dependency of fay)
[__1] rejecting: base-4.12.0.0/installed-4.1... (conflict: fay => base>=4.9 &&
<4.12)
[__1] rejecting: base-4.12.0.0, base-4.11.1.0, base-4.11.0.0, base-4.10.1.0,
...
non-upgradeable package requires installed instance)
[__1] fail (backjumping, conflict set: base, fay)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: fay, base
()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()

After my (relatively minor changes):

()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()
cabal new-build all
Resolving dependencies...
Build profile: -w ghc-8.6.3 -O1
In order, the following will be built (use -v for more details):
- Diff-0.3.4 (lib) (requires download & build)
- charset-0.3.7.1 (lib:charset) (requires download & build)
- data-lens-light-0.1.2.2 (lib) (requires download & build)
- ghc-paths-0.1.0.9 (lib:ghc-paths) (requires download & build)
- mtl-compat-0.2.1.3 (lib) (requires download & build)
- shakespeare-2.0.20 (lib) (requires download & build)
- sourcemap-0.1.6 (lib) (requires download & build)
- traverse-with-class-1.0.0.0 (lib) (requires download & build)
- language-ecmascript-0.19 (lib) (requires download & build)
- fay-0.24.0.1 (lib:fay, exe:fay) (configuration changed)
- fay-base-0.21.1.0 (lib) (first run)
...
Installing language-ecmascript-0.19 (lib)
Completed language-ecmascript-0.19 (lib)
Linking /home/henry/shared/haskell/fay-hhl/dist-newstyle/build/x86_64-linux/ghc-8.6.3/fay-0.24.0.1/setup/setup ...
Configuring fay-0.24.0.1...
Preprocessing library for fay-0.24.0.1..
Building library for fay-0.24.0.1..
[ 4 of 56] Compiling Fay.Compiler.ModuleT ( src/Fay/Compiler/ModuleT.hs, /home/henry/shared/haskell/fay-hhl/dist-newstyle/build/x86_64-linux/ghc-8.6.3/fay-0.24.0.1/build/Fay/Compiler/ModuleT.o ) [Data.Map changed]
...
[22 of 23] Compiling Control.Exception ( src/Control/Exception.hs, /home/henry/shared/haskell/fay-hhl/dist-newstyle/build/x86_64-linux/ghc-8.6.3/fay-base-0.21.1.0/build/Control/Exception.o )
[23 of 23] Compiling Unsafe.Coerce ( src/Unsafe/Coerce.hs, /home/henry/shared/haskell/fay-hhl/dist-newstyle/build/x86_64-linux/ghc-8.6.3/fay-base-0.21.1.0/build/Unsafe/Coerce.o )
()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()

Now it runs:

cabal new-exec fay -- --version
fay 0.24.0.1

()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()

But it cannot find fay-base:

cabal new-exec fay -- examples/calc.hs
fay: ghc-pkg describe error:
ghc-pkg: cannot find package fay-base

CallStack (from HasCallStack):
error, called at src/Fay/Compiler/Packages.hs:60:24 in fay-0.24.0.1-inplace:Fay.Compiler.Packages

()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()

This isn't surprising because internally you use ghc-pkg to find the
fay-base package, but new-build does not register packages in the
global or user package list:

henry@xps ~/haskell/fay-hhl (git)-[master] % ghc-pkg list /home/henry/.ghcup/ghc/8.6.3/lib/ghc-8.6.3/package.conf.d
Cabal-2.4.0.1
array-0.5.3.0
base-4.12.0.0
binary-0.8.6.0
bytestring-0.10.8.2
containers-0.6.0.1
deepseq-1.4.4.0
directory-1.3.3.0
filepath-1.4.2.1
ghc-8.6.3
ghc-boot-8.6.3
ghc-boot-th-8.6.3
ghc-compact-0.1.0.0
ghc-heap-8.6.3
ghc-prim-0.5.3
ghci-8.6.3
haskeline-0.7.4.3
hpc-0.6.0.3
integer-gmp-1.0.2.0
libiserv-8.6.3
mtl-2.2.2
parsec-3.1.13.0
pretty-1.1.3.6
process-1.6.3.0
rts-1.0
stm-2.5.0.0
template-haskell-2.14.0.0
terminfo-0.4.1.2
text-1.2.3.1
time-1.8.0.2
transformers-0.5.5.0
unix-2.7.2.2
xhtml-3000.2.2.1
/home/henry/.ghc/x86_64-linux-8.6.3/package.conf.d
Cabal-2.4.1.0

()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()

I am using:

ghc --version
The Glorious Glasgow Haskell Compilation System, version 8.6.3

()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()

which I installed with ghcup:

which ghc
/home/henry/.ghcup/bin/ghc

()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()

Here is the diff between my private version and your current HEAD in
github:

()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()

diff --git a/.travis.yml b/.travis.yml
index 395d0a9..03cb2f8 100644
  --- a/.travis.yml
  +++ b/.travis.yml
  @@ -5,6 +5,7 @@ ghc:
     - "8.0"
     - "8.2"
     - "8.4"
  +  - "8.6"
   install:
     - "travis_retry cabal install 'cpphs >= 1.18.3' 'happy >= 1.19.5'"
     - "cabal install --dry-run -v3"
  diff --git a/cabal.project b/cabal.project
  index cf74081..5326049 100644
  --- a/cabal.project
  +++ b/cabal.project
  @@ -1 +1,11 @@
   packages: . fay-base/
  +constraints:
  +      haskell-src-exts==1.20.*
  +    , zip-archive==0.4
  +    , hackage-security==0.5.3.0
  +    , HTTP==3.12
  +    , async==2.2.1
  +    , resolv==0.1.1.2
  +    , libiserv==8.6.1
  +  
  +
  diff --git a/fay.cabal b/fay.cabal
  index 33e9e98..66ca72d 100644
  --- a/fay.cabal
  +++ b/fay.cabal
  @@ -139,11 +139,11 @@ library
       Language.Haskell.Names.Types
       Paths_fay
     build-depends:
  -      base >= 4.9 && < 4.12
  +      base >= 4.9 && < 4.13
       , base-compat >= 0.10 && < 0.11
       , aeson > 0.6 && < 1.5
       , bytestring >= 0.9 && < 0.11
  -    , containers >= 0.4 && < 0.6
  +    , containers >= 0.4 && < 0.7
       , data-default >= 0.2 && < 0.8
       , data-lens-light == 0.1.*
       , directory >= 1.1 && < 1.4
  diff --git a/src/Fay/Compiler.hs b/src/Fay/Compiler.hs
  index fde2f00..91b38d9 100644
  --- a/src/Fay/Compiler.hs
  +++ b/src/Fay/Compiler.hs
  @@ -121,7 +121,7 @@ compileFileWithSource filepath contents = do
   -- | Compile a parse HSE module.
   compileModuleFromAST :: ([JsStmt], [JsStmt]) -> F.Module -> Compile ([JsStmt], [JsStmt])
   compileModuleFromAST (hstmts0, fstmts0) mod'@Module{} = do
  -  mod@(Module _ _ pragmas _ decls) <- annotateModule Haskell2010 defaultExtensions mod'
  +  ~mod@(Module _ _ pragmas _ decls) <- annotateModule Haskell2010 defaultExtensions mod'
     let modName = unAnn $ F.moduleName mod
     modify $ \s -> s { stateUseFromString = hasLanguagePragmas ["OverloadedStrings", "RebindableSyntax"] pragmas
                      }
  diff --git a/src/Fay/Compiler/Import.hs b/src/Fay/Compiler/Import.hs
  index 025edd3..7e877d3 100644
  --- a/src/Fay/Compiler/Import.hs
  +++ b/src/Fay/Compiler/Import.hs
  @@ -50,7 +50,7 @@ compileWith filepath with compileModule before from = do
         st
         (parseResult (throwError . uncurry ParseError)
                      (\mod' -> do
  -                     mod@(Module _ _ _ imports _) <-
  +                     ~mod@(Module _ _ _ imports _) <-
                          either throwError return =<< io (before F.noI mod')
                        res <- foldr (<>) mempty <$> mapM (compileImport compileModule) imports
                        modify $ \s -> s { stateModuleName = unAnn $ F.moduleName mod }
  diff --git a/src/Fay/Compiler/InitialPass.hs b/src/Fay/Compiler/InitialPass.hs
  index 4c7eb8d..d750239 100644
  --- a/src/Fay/Compiler/InitialPass.hs
  +++ b/src/Fay/Compiler/InitialPass.hs
  @@ -51,7 +51,7 @@ preprocessFileWithSource filepath contents = do
   preprocessAST :: () -> F.Module -> Compile ()
   preprocessAST () mod@(Module _ _ _ _ decls) = do
     -- This can only return one element since we only compile one module.
  -  ([exports],_) <- HN.getInterfaces Haskell2010 defaultExtensions [mod]
  +  ~([exports],_) <- HN.getInterfaces Haskell2010 defaultExtensions [mod]
     modify $ \s -> s { stateInterfaces = M.insert (stateModuleName s) exports $ stateInterfaces s }
     forM_ decls scanTypeSigs
     forM_ decls scanRecordDecls
  diff --git a/src/Fay/Convert.hs b/src/Fay/Convert.hs
  index 4d68f6e..5788344 100644
  --- a/src/Fay/Convert.hs
  +++ b/src/Fay/Convert.hs
  @@ -139,7 +139,7 @@ parseDataOrTuple rec value = result where
   parseTuple :: Data a => GenericParser -> DataType -> Vector Value -> Either String a
   parseTuple rec typ arr =
     case dataTypeConstrs typ of
  -    [cons] -> evalStateT (fromConstrM (do i:next <- get
  +    [cons] -> evalStateT (fromConstrM (do ~(i:next) <- get
                                             put next
                                             value <- lift (Vector.indexM arr i)
                                             lift (rec value))
  @@ -164,7 +164,7 @@ parseObject rec typ obj =
   -- | Make a simple ADT constructor from an object: { "slot1": 1, "slot2": 2} -> Foo 1 2
   makeSimple :: Data a => GenericParser -> HashMap Text Value -> Constr -> Either String a
   makeSimple rec obj cons =
  -  evalStateT (fromConstrM (do i:next <- get
  +  evalStateT (fromConstrM (do ~(i:next) <- get
                                 put next
                                 value <- lift (lookupField obj (Text.pack ("slot" ++ show i)))
                                 lift (rec value))
  @@ -176,7 +176,7 @@ makeRecord :: Data a => GenericParser -> HashMap Text Value -> Constr -> [String
   makeRecord rec obj cons =
     evalStateT $
       fromConstrM
  -      (do key:next <- get
  +      (do ~(key:next) <- get
             put next
             value <- lift (lookupField obj (Text.pack key))
             lift $ rec value)

()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()

I hope this is helpful. I can send you a pull request if you wish.
Best wishes,
Henry Laxen

[swamp-agr]

I was able to build examples with ghc-8.4.4 and cabal new- commands passing --base-path argument to fay with global db entry (in my case ${HOME}/.cabal/share/x86_64-osx-ghc-8.4.4/fay-base-0.21.1.0).

Next time I will try to reproduce it by ghc-8.6.3 with ghcup as you describe.

Thanks,
@swamp-agr

[swamp-agr]

I realised that new-build stores packages with hashes in nix-style in ${HOME}/.cabal/store, e.g.:
${HOME}/.cabal/store/ghc-8.4.1/fy-bs-0.21.1.0-bfb1af4b/.

I'll try to analyze what improvements it is possible to made to automatically determine this directory and load module from library.

[swamp-agr]

Hi @henrylaxen!

The WA for this problem finally discovered.

Workaround

1. Clone this repo locally.

git clone https://github.com/faylang/fay.git
cd ./fay

2. Install fay, fay-base globally.

cabal new-build
cabal new-install --symlink-bindir=$HOME/.cabal/bin

3. Execute installed fay:

$ ~/.cabal/bin/fay --base-path fay-base --package-conf dist-newstyle/packagedb/ghc-8.6.3 examples/dom.hs

Explanation

Within GHC-8.6.x and new cabal-install with its new-build approach, following file should be created locally, in my case:

$ ls -l .ghc.environment.*
-rw-r--r--  1 agr  staff  2969 Mar  2 02:23 .ghc.environment.x86_64-darwin-8.4.1
-rw-r--r--  1 agr  staff  2893 Apr 28 23:57 .ghc.environment.x86_64-darwin-8.6.3

These files contained information about location of both local and global package databases.
In workaround I used the local package-db to get the resulted target file.

Permanent solution

Briefly, I suppose, it is necessary to implement extra check for .ghc.environment.* pattern in some directories around the fay's launch and to determine global package-db location. Then, obtain it and automatically pass it to the fay.

1. Determine the closest .ghc.environment.* file to exec directory.
2. If exists, read global package-db from it.
3. If exists, pass the location to fay compiler.

Thank you Andrey for looking into this. I was working around it by doing the old cabal install instead of new-build. I'm glad you found a way to run fay with the new system. Thanks again.

[swamp-agr]

By the way, GHC 8.6 support is available in fay-0.24.0.3. It's already on Hackage.

Thank you!

Hi Andrey,

Sorry to say, I'm baaaccck. I just upgraded ghc to 8.8.3 and am again having problems installing fay. Here is a script I wrote which which you should be able to reproduce what is happening.

# in your .cabal/config file make sure you have installdir: $HOME/.cabal/bin
# I am using ghc 8.8.3 and ghcup to comple and install ghc, cabal

`
cd $HOME
cat $HOME/fay-install.sh
rm -rf $HOME/.cabal/bin
rm -rf $HOME/.cabal/lib
rm -rf $HOME/.cabal/logs
rm -rf $HOME/.cabal/packages
rm -rf $HOME/.cabal/setup-exe-cache
rm -rf $HOME/.cabal/share
rm -rf $HOME/.ghc/*
rm -rf $HOME/fay
cd $HOME/ghcup ; ghcup install ; ghcup install-cabal ; cabal update ; cabal install cabal-install
cd $HOME
ghc --version ; cabal --version
cabal v2-install cpphs happy
git clone https://github.com/faylang/fay.git
cd $HOME/fay ; cabal v2-install ; cabal v2-build fay-base
ls -ahl $HOME/fay/dist-newstyle/packagedb/ghc-8.8.3
~/.cabal/bin/fay --base-path fay-base --package-conf dist-newstyle/packagedb/ghc-8.8.3 examples/dom.hs

`

You can see the end result here.

Sadly it ends with:

`+ /home/henry/.cabal/bin/fay --base-path fay-base --package-conf dist-newstyle/packagedb/ghc-8.8.3 examples/dom.hs
fay: ghc: : cannot satisfy -package fay-base-0.21.1.1:
fay-base-0.21.1.1-inplace is unusable due to missing dependencies:
fay-0.24.1.0-inplace
(use -v for more information)

CallStack (from HasCallStack):
error, called at src/Fay.hs:61:19 in fay-0.24.1.0-f4db8500ad2134d7790109cabfd935b17015992b00db6366343b4838048ce90d:Fay
bash -x fay-install.sh 87.47s user 7.22s system 81% cpu 1:56.33 total
`
I have tried several things to get around this, but so far no joy.
Best wishes,
Henry Laxen

[bneijt]

Workaround for me, for now, is to use docker to run fay in an older environment. I've posted a docker image on dockerhub

[swamp-agr]

Hi @henrylaxen, hope you are doing great!

I investigated further new-install or v2-install commands.

As of now, please use v1-install command, this way you can be sure, that fay-base sources will be moved to $HOME/.cabal/share/$ARCH-$OS-ghc-$GHCVERSION/fay-base-X.X.X.X/src directory and could be registered in local package database and accessible further via current fay mechanics (ghc-pkg lookup).

• With v2-build or v2-install commands only library produced and stored inside Cabal Store (i.e. $HOME/.cabal/store) that is alternative to package db and v1 approach. Thus, without sources, --base-path should be always specified.
• Moreover, ghc-pkg could locate fay and fay-base packages in a broken state.
• I consulted with Cabal Team about issues because it looks really strange.
• They pointed out to package environments which I want to investigate further.

Separate discussion: #469.

Thanks,
@swamp-agr

[robbiemu]

what should I do if after installing it with cabal install fay fay-base, I am seeing ~/.cabal/store/ghc-8.10.5/fy-bs-0.21.2.0-01594354/share/src/, there is no ~/.cabal/share folder, and:

$ ghc-pkg describe fay-base
ghc-pkg: cannot find package fay-base

this is a fresh install of the environment so its.. probable that I did something wrong before getting here I guess.

I also don't see it in cabal list, but I think this is a known bug:

MacBook-Pro-2:~ robertotomas$ cabal list fay-base
* fay-base
    Synopsis: The base package for Fay.
    Default available version: 0.21.2.0
    Installed versions: [ Not installed ]
    Homepage: https://github.com/faylang/fay/
    License:  BSD3

With cabal-install 3.0 and beyond, packages you "install" are put into the store rather than the packagedb, and then made available through tools by being listed in the global ghc environment file. The list command isn't updated to be aware of this workflow, and only lists packages which are in the standard packagedb.
-- src https://stackoverflow.com/questions/57932348/cabal-install-works-but-cabal-list-does-not-find-package

I do see the fy-bs file listed there:

/Users/robertotomas/.ghc/x86_64-darwin-8.10.5/environments/default

package-id fy-bs-0.21.2.0-01594354

and that is in my /UseLLY,rs/robertotomas/.cabal/store/ghc-8.10.5/package.db/

finally, I can get a bit further if I try to load the share folder under store/fs-bs...:

$ fay --base-path /Users/robertotomas/.cabal/store/ghc-8.10.5/fy-bs-0.21.2.0-01594354/share --package-conf /Users/robertotomas/.cabal/store/ghc-8.10.5/package.db Young.hs 
fay: ghc: 
Young.hs:10:1: error:
    Could not load module ‘Control.Monad’
    It is a member of the hidden package ‘base-4.14.2.0’.
    You can run ‘:set -package base’ to expose it.
    (Note: this unloads all the modules in the current scope.)
    Use -v (or `:set -v` in ghci) to see a list of the files searched for.
   |
10 | import Control.Monad
   | ^^^^^^^^^^^^^^^^^^^^

CallStack (from HasCallStack):
  error, called at src/Fay.hs:61:19 in fy-0.24.

and with a Main.hs from a cabal init:

$ fay --base-path /Users/robertotomas/.cabal/store/ghc-8.10.5/fy-bs-0.21.2.0-01594354/share --package-conf /Users/robertotomas/.cabal/store/ghc-8.10.5/package.db app/Main.hs 
fay: ghc: 
app/Main.hs:3:9: error:
    Not in scope: type constructor or class ‘IO’
  |
3 | main :: IO ()
  |         ^^

CallStack (from HasCallStack):
  error, called at src/Fay.hs:61:19 in fy-0.24.2.0-8b43beec:Fay

[nshinchan]

ghcup install stack 3.3.1

ghcup install cabal 3.12.1.0

stack install fay --compiler 8.8.3 <- failure. but installed ghc-8.8.3

cabal install fay -w ghc-8.8.3