# JwstCli

A cli wrapper and downloader built around jwstapi.com

## Author

raf+jwst@dreamthought.com

## Warning and Context

- This is a hobby project which is being used to learn elixir's OTP, whilst scratching an itch to explore
the new [jwstapi.com](jwstapi.com).
- This API is to accompany dreamily staring into nebuli.
- Use with caution.

## Future work

`JwstCli.Api` may get factored out for reuse as a stand alone library, which wraps HTTPoison.

## Tests

Will grow as more API endpoints and the project matures.

# Installation

### Local

1. Clone
2. `mix release`

### Hex (not yet set up)

If [available in Hex](https://hex.pm/docs/publish), the package can be installed
by adding `jwst_cli` to your list of dependencies in `mix.exs`:

```elixir
def deps do
  [
    {:jwst_cli, "~> 0.1.0"}
  ]
end
```

# Documentation

## Generating
Documentation can be generated with [ExDoc](https://github.com/elixir-lang/ex_doc)
and published on [HexDocs](https://hexdocs.pm). Once published, the docs can
be found at <https://hexdocs.pm/jwst_cli>.


## Prod

Let's not go there yet. But do contribute back if you do.



# Execution

1. Set JWST_API_KEY in your shell. Eg. `export JWST_API_KEY=abcdKitten1234`
2. Run mix release
3. As below, start the release in `start_iex`, or with `daemon` and `remote`


## Set JWST_API_KEY

You can obtain a new API key from [jwstapi.com](https://jwstapi.com)

This provided from the environment and baked into the release at build time.
It may be customised per environment using (./config/config.exs)[./config/config.exs] and sibling overides.

Eg.

```bash
  export JWST_API_KEY=abcdKitten1234
```

## Buiding the Release

We simply run `mix release`

```bash
  mix release
Compiling 5 files (.ex)
warning: module attribute @me was set but never used
  lib/jwst_cli/repl/executor.ex:9

Generated jwst_cli app
Release jwst_cli-0.1.0 already exists. Overwrite? [Yn] y
* assembling jwst_cli-0.1.0 on MIX_ENV=dev
* skipping runtime configuration (config/runtime.exs not found)
* skipping elixir.bat for windows (bin/elixir.bat not found in the Elixir installation)
* skipping iex.bat for windows (bin/iex.bat not found in the Elixir installation)

Release created at _build/dev/rel/jwst_cli!
```

## Connecting to the App to query JWST API

As mentioned there are two methods

### Running as a deamon and connecting remotely

This scenario is better suited to a long running scenario where you may want to connect remotely.

#### Start the app as a background daemon

This will leave the GenServer (with supervisor) running in the background


```elixir
   _build/dev/rel/jwst_cli/bin/jwst_cli daemon
```

This does not produce any output.

#### Connect Remotely

Very similar but you pass in `remote`. This pattern allows you abort each iex session, but still connect
back to the same daemon to query the API.

Eg. The following connects and queries the list of JWST program id's:

```elixir
 _build/dev/rel/jwst_cli/bin/jwst_cli remote
Erlang/OTP 25 [erts-13.0.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [jit:ns]

Interactive Elixir (1.13.2) - press Ctrl+C to exit (type h() ENTER for help)
 
iex(jwst_cli@feynman)1> GenServer.call(JwstCli.Repl.Executor, {:execute, "get_program_list"})
[2731, 2732, 2733, 2734]
```

See also: 
[Elixir Mix Release by Example](https://zemuldo.com/blog/elixir-mix-releases-by-example---powerful-and-inbuilt-6129462ad2bd8290436304ca)

## start_iex

This creates a _blocking_ process which will terminate on exit from `iex`.

It also produces greater visibility of logging output from the GenServer.

Eg.

```elixir
  _build/dev/rel/jwst_cli/bin/jwst_cli start_iex
Erlang/OTP 25 [erts-13.0.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [jit:ns]


10:16:22.183 [info]  Using Children

10:16:22.183 [info]  [{JwstCli.Repl.Executor, [%{api_key: "**************"}]}]

10:16:22.183 [info]  [%{api_key: "***************"}]

10:16:22.183 [info]  Start on #PID<0.905.0>

10:16:22.183 [info]  [active: 1, specs: 1, supervisors: 0, workers: 1]
Interactive Elixir (1.13.2) - press Ctrl+C to exit (type h() ENTER for help)
iex(jwst_cli@feynman)1> GenServer.call(JwstCli.Repl.Executor, {:execute, "get_program_list"})
10:16:49.465 [info]  "get_program_list"
10:16:49.466 [debug] Invoking /program/list with *********
10:16:50.605 [info]  [2731, 2732, 2733, 2734]
[2731, 2732, 2733, 2734]``
```

# GenServer calls for Querying the API

Once you're in a session, you can query the GenServer by using `call` with the following general form:

```elixir
iex> GenServer.call(JwstCli.Repl.Executor, {:execute, "<name of operation>"})
```

Operations with names ending in `_raw` will typically return the raw content of the equivalent remote resources
as JSON (wrapped in a JASON.Response).

See below for you choice of operations

## get_program_list

Returns all JWST program ids as an array

Eg. 

```elixir
iex> GenServer.call(JwstCli.Repl.Executor, {:execute, "get_program_list"})
[2731, 2732, 2733, 2734]        
```

## get_program_list_raw

Returns all JWST program ids contained in the response from /program/list 
[defined by the JWST API](https://documenter.getpostman.com/view/10808728/UzQyphjT#14929ed7-0b6a-4966-98d6-3062b74e8e04)

Eg. 

```elixir
GenServer.call(JwstCli.Repl.Executor, {:execute, "get_program_list_raw"})
```
## get_recent_jpg

Returns last 30 jpgs
[defined by the JWST API](https://documenter.getpostman.com/view/10808728/UzQyphjT#a0b2c1b0-d2e7-46b0-bb19-20df18c94f09)

Eg. 

```elixir
GenServer.call(JwstCli.Repl.Executor, {:execute, "get_recent_jpg"})
iex(jwst_cli@feynman)6> GenServer.call(JwstCli.Repl.Executor, {:execute, "get_recent_jpg"})
```


## get_all_jpg

Returns all JWST jpegs URLs from JWST
[defined by the JWST API](https://documenter.getpostman.com/view/10808728/UzQyphjT#a0b2c1b0-d2e7-46b0-bb19-20df18c94f09)

Eg. 

```elixir
GenServer.call(JwstCli.Repl.Executor, {:execute, "get_all_jpg"})
iex(jwst_cli@feynman)6> GenServer.call(JwstCli.Repl.Executor, {:execute, "get_all_jpg"})
```

## get_all_jpg_raw

Returns all JWST jpegs from JWST
[defined by the JWST API](https://documenter.getpostman.com/view/10808728/UzQyphjT#a0b2c1b0-d2e7-46b0-bb19-20df18c94f09)

Eg. 

```elixir
GenServer.call(JwstCli.Repl.Executor, {:execute, "get_all_jpg_raw"})
iex(jwst_cli@feynman)6> GenServer.call(JwstCli.Repl.Executor, {:execute, "get_all_jpg_raw"})

{:ok,
 %HTTPoison.Response{
   body: "{\"statusCode\":200,\"body\":[{\"id\":\"jw02731002001_02107_00004_mirimage_o002_crf_thumb.jpg\",\"observation_id\":\"jw02731002001_02107_00004_mirimage_o002\",\"program\":2731,\"details\":{\"mission\":\"JWST\",\"instruments\":[{\"instrument\":\"FGS\"},{\"instrument\":\"NIRCam\"},{\"instrument\":\"NIRISS\"},{\"instrument\":\"NIRSpec\"},{\"instrument\":\"MIRI\"}],\"suffix\":\"_thumb\",\"description\":\"thumbnail image of the FITS data product\"},\"file_type\":\"jpg\",\"thumbnail\":\"\",\"location\":\"https://stpubdata-jwst.stsci.edu/ero/jw02731/jw02731002001/jw02731002001_02107_00004_mirimage_o002_crf_thumb.jpg\"], ....
   headers: [...],
   request: %HTTPoison.Request{ .... },
   request_url: ....,
   status_code: 200
 }}
```

## get_all_fits_raw

Returns all JWST FITS imagse from JWST
[defined by the JWST API](https://documenter.getpostman.com/view/10808728/UzQyphjT#a0b2c1b0-d2e7-46b0-bb19-20df18c94f09)

Eg. 

```elixir
GenServer.call(JwstCli.Repl.Executor, {:execute, "get_all_fits_raw"})
iex(jwst_cli@feynman)6> GenServer.call(JwstCli.Repl.Executor, {:execute, "get_all_fits_raw"})

{:ok,
 %HTTPoison.Response{
   body: "{\"statusCode\":200,\"body\":[{\"id\":\"jw02731001003_02105_00001_nrca1_i2d.fits\",\"observation_id\":\"jw02731001003_02105_00001_nrca1\",\"program\":2731,\"details\":{\"mission\":\"JWST\",\"instruments\":[{\"instrument\":\"FGS\"},{\"instrument\":\"NIRCam\"},{\"instrument\":\"NIRISS\"},{\"instrument\":\"NIRSpec\"},{\"instrument\":\"MIRI\"}],\"suffix\":\"_i2d\",\"description\":\"exposure/target (L2b/L3): rectified 2D image\"},\"file_type\":\"fits\",\"thumbnail\":\"\",\"location\":\"https://stpubdata-jwst.stsci.edu/ero/jw02731/jw02731001003/jw02731001003_02105_00001_nrca1_i2d.fits\",...
   headers: [...],
   request: %HTTPoison.Request{ .... },
   request_url: ....,
   status_code: 200
 }}
```

## get_all_ecsv_raw

Returns all ecsv file url's from JWST
[defined by the JWST API](https://documenter.getpostman.com/view/10808728/UzQyphjT#a0b2c1b0-d2e7-46b0-bb19-20df18c94f09)

Eg. 

```elixir
GenServer.call(JwstCli.Repl.Executor, {:execute, "get_all_ecsv_raw"})
iex(jwst_cli@feynman)6> GenServer.call(JwstCli.Repl.Executor, {:execute, "get_all_ecsv_raw"})

{:ok,
 %HTTPoison.Response{
   body: "{\"statusCode\":200,\"body\":[{\"id\":\"jw02731-o001_t017_nircam_clear-f444w_cat.ecsv\",\"observation_id\":\"jw02731-o001_t017_nircam_clear-f444w\",\"program\":2731,\"details\":{\"mission\":\"JWST\",\"instruments\":[{\"instrument\":\"FGS\"},{\"instrument\":\"NIRISS\"}],\"suffix\":\"_cat\",\"description\":\"target (L3) : source catalog\"},\"file_type\":\"ecsv\",\"thumbnail\":\"\",\"location\":\"https://stpubdata-jwst.stsci.edu/ero/jw02731/L3/t/jw02731-o001_t017_nircam_clear-f444w_cat.ecsv\"},{\"id\":\"jw02731-o001_t017_nircam_f444w-f470n_cat.ecsv\",\"observation_id\":\"jw02731-o001_t017_nircam_f444w-f470n\",\"program\":2731,\"details\":{\"mission\":\"JWST\",\"instruments\":[{\"instrument\":\"FGS\"},{\"instrument\":\"NIRISS\"}],\"suffix\":\"_cat\",\"description\":\"target (L3) : source catalog\"},\"file_type\":\"ecsv\",\"thumbnail\":\"\",\"location\":\"https://stpubdata-jwst.stsci.edu/ero/jw02731/L3/t/jw02731-o001_t017_nircam_f444w-f470n_cat.ecsv\",...
   headers: [...],
   request: %HTTPoison.Request{ .... },
   request_url: ....,
   status_code: 200
 }}
```

## get_all_json_raw

Returns all json file url's from JWST
[defined by the JWST API](https://documenter.getpostman.com/view/10808728/UzQyphjT#a0b2c1b0-d2e7-46b0-bb19-20df18c94f09)

Eg. 

```elixir
GenServer.call(JwstCli.Repl.Executor, {:execute, "get_all_json_raw"})
iex(jwst_cli@feynman)6> GenServer.call(JwstCli.Repl.Executor, {:execute, "get_all_json_raw"})

{:ok,
 %HTTPoison.Response{
   body: "{\"statusCode\":200,\"body\":[{\"id\":\"jw02731-o001_20220712t165318_image2_317_asn.json\",\"observation_id\":\"jw02731-o001_20220712t165318_image2_317\",\"program\":2731,\"details\":{\"mission\":\"JWST\",\"instruments\":[{\"instrument\":\"FGS\"},{\"instrument\":\"NIRCam\"},{\"instrument\":\"NIRISS\"},{\"instrument\":\"NIRSpec\"},{\"instrument\":\"MIRI\"}],\"suffix\":\"_asn\",\"description\":\"source/target (L3) : association generator\"},\"file_type\":\"json\",\"thumbnail\":\"\",\"location\":\"https://stpubdata-jwst.stsci.edu/ero/jw02731/asn/jw02731-o001_20220712t165318_image2_317_asn.json\"}, ...
   headers: [...],
   request: %HTTPoison.Request{ .... },
   request_url: ....,
   status_code: 200
 }}
```

# Log Levels

The app is currently VERY chatting. You may want to reduce the default log levels; this is on my TODO.

In iex:

```elixir
iex> Logger.configure(level: :error)
```

In the `config/config.exs` 
```elixir
config :logger, level: :error
```

# Stand alone session (WIP)

## Buiding

This is a work in progres and will ulimately provide a more powerful standalone repl.
Run the following to generate the jwst_cli executable in the parent directory.
```elixir
mix escript.build
```

Still in progress - this will be a cli app to query the GenServer session as a connected node.
You could potentially run this by starting iex with `elixir --cookie <common value> and -s script`
I've yet to document this properly.

# Thanks

Thank you to [Kyle Redelinghuys](https://ksred.com/) who has provided the JWST API