Data Science Stack Exchange
Data Science Stack Exchange

Questions tagged [automatic-summarization]

31 questions
9
votes
1 answer

How to summarize a long text using GPT-3

What is the best way to summarize a long text that exceeds 4096 token limit (like a podcast transcript for example)? As I understand I need to split the text into chunks to summarize, and then concatenate the results and summarize those. Is there…
Poma
  • 193
  • 1
  • 5
5
votes
1 answer

How to use df.groupby() to select and sum specific columns w/o pandas trimming total number of columns

I got Column1, Column2, Column3, Column4, Column5, Column6 I'd like to group Column1 and get the row sum of Column3,4 and 5 When I apply groupby() and get this that is correct but it's leaving out Column6: df =…
Steven
  • 119
  • 2
  • 4
  • 15
4
votes
1 answer

Detect sensitive data from unstructured text documents

I know this question is broad, but I need an advice to know if it's possible to achieve what I want to do. The problem is that I have around 2500 documents with sensitive data being replaced by four dots. I do not have the original documents, so I…
user971961
  • 41
  • 2
3
votes
0 answers

Dataset availability for automatic text summarization

I'm working on an automatic text summarization NLP problem and looking for a dataset with USA legal case reports similar to the Australian legal case reports dataset in UCI repository. Can you please refer me to any such dataset? I've not been able…
VishwaV
  • 379
  • 3
  • 6
  • 18
3
votes
0 answers

summarizing time series dataset: extract time window sliding, change points, pattern seasonality in time series

I need to detect list of change points in time series dataset (temperature), and I need to split dataset into set of classes (patterns) and detect seasonality of each class (pattern). for example suppose that we create 2-classes named called and…
Mina Younan
  • 31
  • 3
3
votes
7 answers

Should I use regex or machine learning?

I am thinking of two use cases: You submit a resume in PDF format to a web site and it extracts your contact information, job titles, etc. You receive an email from a friend that says, "let's have lunch next Tuesday" and your email program detects…
I_Play_With_Data
  • 2,079
  • 2
  • 16
  • 39
2
votes
0 answers

Calculate a mean on condtions in a dataframe with dyplr

My table looks like this: Tissue Dry Amount Analyte Area 1 Liver A a 3-Phosphoglyceric Acid 66351918.4 2 Liver B a 3-Phosphoglyceric Acid 119013081.6 3 Liver A b 3-Phosphoglyceric Acid 195732464.0 4 …
Christina Lucas
  • 21
  • 1
2
votes
1 answer

Is there a dataset with news articles and their headlines?

I need a set of news headlines and articles to help me in a project on automatic summarization. Is there such a dataset or something similar?
mosafattah
  • 21
  • 2
2
votes
0 answers

What other approaches are there for abstractive summarization, other then seq2seq?

I'm researching on abstractive text summarization, and has come across many recent papers. They all seem to be focusing on Sequence to Sequence models based on RNNs. Apart from RNNs, what other approaches are there when it comes to abstractive text…
Dee
  • 21
  • 2
2
votes
2 answers

Extractive text summarization, as a classification problem using deep networks

I understand that this is broad, but I merely require a few pointers. I wish to implement a text summarization system. I have already tried out an abstractive approach, now I am trying an extractive approach. The current extractive approaches out…
darthvader1996
  • 71
  • 4
2
votes
2 answers

Extract sentences from beginning of news in single document summarization

I am working on Single Document Summarization task on News datasets. I do some experiments in this task. A simple experiment that I make and has a good result is extracting sentences just from beginning of news. Now I want to find any paper or…
Mahmood Kohansal
  • 21
  • 4
1
vote
0 answers

Huggingface Library - Multi-document summarization

Can BART, PEGASUS ... etc. API in huggingface library be used to directly perform multi document summarization? (e.g. here: https://huggingface.co/transformers/model_doc/bart.html)
asahi kibou
  • 143
  • 1
  • 5
1
vote
1 answer

UniLM - Unified Language Model for summarization

The UniLM claims to be the best approach for summarization task. But there doesn't seem to be any tutorial or how-to section in the README.md or any other blog. How exactly can I use this state-of-the-art library for abstractive summary…
Nabin
  • 113
  • 5
1
vote
2 answers

Text summarization with limited number of words

I am reviewing summarization techniques and haven't (yet) found an approach to limit the length of a summary. So for example a summarization function that gives me a summary that is < 500 words. Can you point me in the right direction? Are there…
Carol.Kar
  • 187
  • 5
1
vote
0 answers

Generate sentences using given data

I am working on an automated insights generation use case where I want to generate meaningful sentences from given aggregated data. For example, Data: Student = John Total_Marks = 96 Class_Average = 85 NLG model-generated insights: 1. You did an…
ProgramSpree
  • 143
  • 1
  • 6
1
2 3