LSTM Neural Networks for Spectroscopic Studies of Type Ia Supernovae

Last update: Oct 11, 2022

Overview

Package Description

The difficulties in acquiring spectroscopic data have been a major challenge for supernova surveys. snlstm is developed to provide a data-driven solution. Based on an observation dataset including 3091 spectra from 361 individual SNe Ia, we trained LSTM neural networks to learn from the spectroscopic time-series data of type Ia supernovae. The model enables the construction of spectral sequences from spectroscopic observations with very limited time coverage.

This repository is associated to the paper "Spectroscopic Studies of Type Ia Supernovae Using LSTM Neural Networks (Hu et al. 2021, ApJ, under review)".

Installation

One can install any desired version of snlstm from Github https://github.com/thomasvrussell/snlstm:

python setup.py install

Additional dependencies

R : In order to reduce the data dimension, we use Functional Principal Component Analysis (FPCA) to parameterize supernova spectra before feeding them into neural networks. The FPCA parameterization and FPCA reconstruction are achieved by the fpca package in R programming language. One can install them, e.g., on CentOS
```
$ yum install R
R > install.packages("fpca")
```
TensorFlow : tensorflow is required to load a given LSTM model and make the spectral predictions. The default LSTM model in this repository is trained on an enviornment with tensorflow 1.14.0. To avoid potential incompatiability issues casued by different tensorflow versions, we recommend users to install the same version via Conda
```
conda install -c anaconda tensorflow=1.14.0
```
PYPHOT (optional) : pyphot is a portable package to compute synthetic photometry of a spectrum with given filter. In our work, the tool was used to correct the continuum component of a supernova spectrum so that its synthetic photometry could be in line with the observed light curves. One may consider to install the package if such color calibration is necessary. We recommend users to install the latest version from Github (pyphot 1.1)
```
pip install git+https://github.com/mfouesneau/pyphot
```

Download archival datasets

snlstm allows users to access to the following archival datasets

[1]

A spectral-observation dataset : it is comprised of 3091 observed spectra from 361 SNe Ia, largely contributed from CfA (Blondin et al. 2012), BSNIP (Silverman et al. 2012), CSP (Folatelli et al. 2013) and Supernova Polarimetry Program (Wang & Wheeler 2008; Cikota et al. 2019a; Yang et al. 2020).

[2]	A spectral-template dataset : it includes 361 spectral templates, each of them (covering -15 to +33d with wavelength from 3800 to 7200 A) was generated from the available spectroscopic observations of an individual SN via a LSTM neural network model.

[3]	An auxiliary photometry dataset : it provides the B & V light curves of these SNe (in total, 196 available), that were used to calibrate the synthetic B-V color of the observed spectra.

These datasets are stored on Zenodo platform, one can download the related files (~ 2GB) through the Zenodo page: https://doi.org/10.5281/zenodo.5637790.

Quick start guide

We prepared several jupyter notebooks as quick tutorials to use our package in a friendly way.

[*]	1-Access_to_Archival_ObservationData.ipynb : this notebook is to show how to access to the spectral-observation dataset and the auxiliary photometry dataset.

[†]	2-Access_to_Archival_TemplateData.ipynb : one can obtain the LSTM generated spectral time sequences in the spectral-template dataset following this notebook.

[‡]	3-SpecData_Process_Example.ipynb : the notebook demonstrates the pre-processing of the spectroscopic data described in our paper, including smooth, rebinning, lines removal and color calibration, etc.

[§]	4-LSTM_Predictions_on_New_SN.ipynb : the notebook provides a guide for users who want apply our LSTM model on very limited spectroscopic data of newly discovered SNe Ia. In this notebook, we use SN 2016coj, a well-observed SN Ia from the latest BSNIP data release, as an example.

[¶]

5-LSTM_Estimate_Spectral_Phase.ipynb : our neural network is trained based on the spectral data with known phases, however, it is still possible to apply the model to the spectra without any prior phase knownlege. The idea is wrong given phase of input spectrum will degrade the predictive accuracy of our method, that is to say, we can find the best-fit phase of input spectrum by minimizing the accuacy of prediction for itself. This notebook is to show how to estimate spectral phase via our model. For the case of SN 2016coj in the notebook, the estimation errors are around 0.5 - 2.0d.

Publications use our method

SN2018agk: A prototypical Type Ia Supernova with a smooth power-law rise in Kepler (K2) (Qinan Wang, et al., 2021, ApJ, see Figure 5 & 6).

Todo list

Support spectral sequence with arbitrary timesteps as input. (current model only accepts spectral pair inputs.)
Support more flexible wavelength range for input spectra. (current model is trained on spectra with uniform wavelength range from 3800 to 7200 A.)

Common issues

TBD

Development

The latest source code can be obtained from https://github.com/thomasvrussell/snlstm.

When submitting bug reports or questions via the issue tracker, please include the following information:

OS platform.
Python version.
Tensorflow version.
Version of snlstm.

Cite

Spectroscopic Studies of Type Ia Supernovae Using LSTM Neural Networks (Hu et al. 2021, ApJ, under review).

Incorporating Transformer and LSTM to Kalman Filter with EM algorithm

Deep learning based state estimation: incorporating Transformer and LSTM to Kalman Filter with EM algorithm Overview Kalman Filter requires the true p

57 Dec 27, 2022

Forecasting directional movements of stock prices for intraday trading using LSTM and random forest

Forecasting directional movements of stock-prices for intraday trading using LSTM and random-forest https://arxiv.org/abs/2004.10178 Pushpendu Ghosh,

270 Dec 24, 2022

Deep learning based hand gesture recognition using LSTM and MediaPipie.

Hand Gesture Recognition Deep learning based hand gesture recognition using LSTM and MediaPipie. Demo video using PingPong Robot Files Pretrained mode

24 Nov 11, 2022

A3C LSTM Atari with Pytorch plus A3G design

NEWLY ADDED A3G A NEW GPU/CPU ARCHITECTURE OF A3C FOR SUBSTANTIALLY ACCELERATED TRAINING!! RL A3C Pytorch NEWLY ADDED A3G!! New implementation of A3C

532 Jan 2, 2023

Tree LSTM implementation in PyTorch

Tree-Structured Long Short-Term Memory Networks This is a PyTorch implementation of Tree-LSTM as described in the paper Improved Semantic Representati

529 Dec 10, 2022

Sign Language is detected in realtime using video sequences. Our approach involves MediaPipe Holistic for keypoints extraction and LSTM Model for prediction.

RealTime Sign Language Detection using Action Recognition Approach Real-Time Sign Language is commonly predicted using models whose architecture consi

15 Aug 20, 2022

Releases(v1.1.2)

v1.1.2(May 15, 2022)

SuperNova Artificial Inference by Lstm neural networks (SNAIL), stable version 1.1.2.
Source code(tar.gz)
Source code(zip)
v1.0.7(Feb 8, 2022)

SuperNova Artificial Inference by Lstm neural networks (SNAIL), stable version 1.0.7.
Source code(tar.gz)
Source code(zip)
v1.0.6(Feb 8, 2022)

SuperNova Artificial Inference by Lstm neural networks (SNAIL), version 1.0.6. Minor updates on tutorial notebooks.
Source code(tar.gz)
Source code(zip)
v1.0.5(Jan 15, 2022)

SuperNova Artificial Inference by Lstm neural networks (SNAIL), version 1.0.5. Package name has been fixed. We also made minor corrections and updates.
Source code(tar.gz)
Source code(zip)
v1.0.4(Jan 5, 2022)

Allows for estimations on spectral phase using two spectra and other minor upgrades.
Source code(tar.gz)
Source code(zip)
v1.0.3(Nov 23, 2021)

Minor Upgrades
Source code(tar.gz)
Source code(zip)
v1.0.2(Nov 6, 2021)

Add new module to predict phase from a single observed spectrum.
Source code(tar.gz)
Source code(zip)
v1.0.1(Nov 5, 2021)

version 1.0.1
Source code(tar.gz)
Source code(zip)

LSTM Neural Networks for Spectroscopic Studies of Type Ia Supernovae

Related tags

Overview

Package Description

Installation

Additional dependencies

Download archival datasets

Quick start guide

Publications use our method

Todo list

Common issues

Development

Cite

You might also like...

Incorporating Transformer and LSTM to Kalman Filter with EM algorithm

Forecasting directional movements of stock prices for intraday trading using LSTM and random forest

Deep learning based hand gesture recognition using LSTM and MediaPipie.

A3C LSTM Atari with Pytorch plus A3G design

Tree LSTM implementation in PyTorch

Sign Language is detected in realtime using video sequences. Our approach involves MediaPipe Holistic for keypoints extraction and LSTM Model for prediction.

LSTM and QRNN Language Model Toolkit for PyTorch

Multi-layer convolutional LSTM with Pytorch

A Simple LSTM-Based Solution for "Heartbeat Signal Classification and Prediction" in Tianchi

Releases(v1.1.2)

v1.1.2(May 15, 2022)

v1.0.7(Feb 8, 2022)

v1.0.6(Feb 8, 2022)

v1.0.5(Jan 15, 2022)

v1.0.4(Jan 5, 2022)

v1.0.3(Nov 23, 2021)

v1.0.2(Nov 6, 2021)

v1.0.1(Nov 5, 2021)

Owner

GeoTransformer - Geometric Transformer for Fast and Robust Point Cloud Registration

Cross-lingual Transfer for Speech Processing using Acoustic Language Similarity

Repo for EchoVPR: Echo State Networks for Visual Place Recognition

Head and Neck Tumour Segmentation and Prediction of Patient Survival Project

Test-Time Personalization with a Transformer for Human Pose Estimation, NeurIPS 2021

Codes for our paper The Stem Cell Hypothesis: Dilemma behind Multi-Task Learning with Transformer Encoders published to EMNLP 2021.

SigOpt wrappers for scikit-learn methods

NeurIPS-2021: Neural Auto-Curricula in Two-Player Zero-Sum Games.

State-of-the-art language models can match human performance on many tasks

[SDM 2022] Towards Similarity-Aware Time-Series Classification

A repository built on the Flow software package to explore cyber-security attacks on intelligent transportation systems.

Pytorch code for "Text-Independent Speaker Verification Using 3D Convolutional Neural Networks".

PyTorch Code for NeurIPS 2021 paper Anti-Backdoor Learning: Training Clean Models on Poisoned Data.

An unofficial implementation of "Unpaired Image Super-Resolution using Pseudo-Supervision." CVPR2020

Conservative and Adaptive Penalty for Model-Based Safe Reinforcement Learning

Direct design of biquad filter cascades with deep learning by sampling random polynomials.

This is a TensorFlow implementation for C2-Rec

Official PyTorch code of DeepPanoContext: Panoramic 3D Scene Understanding with Holistic Scene Context Graph and Relation-based Optimization (ICCV 2021 Oral).

University of Rochester 2021 Summer REU focusing on music sentiment transfer using CycleGAN

We evaluate our method on different datasets (including ShapeNet, CUB-200-2011, and Pascal3D+) and achieve state-of-the-art results, outperforming all the other supervised and unsupervised methods and 3D representations, all in terms of performance, accuracy, and training time.