预处理以消除连字符而不是连字符

我想做什么

对于NLP预处理,我想摆脱连字符而不是破折号。

输入项

samples = [
    'A former employee of the accused company, ———, offered a statement off the record.', #three dashes
    'He is afraid of two things — spiders and senior prom.' #dash
    'Fifty-six bottles of pop on the wall, fifty-six bottles of pop.' #hyphen
]

预期产量

#output
['A former employee of the accused company','offered a statement off the record.']
['He is afraid of two things', 'spiders and senior prom.']
['Fifty-six bottles of pop on the wall', 'fifty-six bottles of pop.']

以上句子来自以下有关连字符和破折号的两篇文章。

问题

  1. 除去符号“-”的第一个过程失败,并且很难理解为什么第二句和第三句在没有单引号的情况下合并的原因。
#output
['A former employee of the accused company, — — —, offered a statement off the record.', 
'He is afraid of two things—spiders and senior prom.
Fifty-six bottles of pop on the wall, fifty-six bottles of pop.']
  1. 我不知道如何编写区分连字符和破折号的代码。

当前代码

samples = [
    'A former employee of the accused company, — — —, offered a statement off the record.', #dash
    'He is afraid of two things—spiders and senior prom.' #dash
    'Fifty-six bottles of pop on the wall, fifty-six bottles of pop.' #hyphen
]

ignore_symbol = ['-']
for i in range(len(samples)):
    text = samples[i]
    ret = []
    for word in text.split(' '):
        ignore = len(word) <= 0 
        for iw in ignore_symbol:
            if word == iw:
                ignore = True
                break
        if not ignore:
            ret.append(word)

    text = ' '.join(ret)
    samples[i] = text
print(samples)

#output
['A former employee of the accused company, — — —, offered a statement off the record.', 
'He is afraid of two things—spiders and senior prom.
Fifty-six bottles of pop on the wall, fifty-six bottles of pop.']

for i in range (len(samples)):
    list_temp = []
    text = samples[i]
    list_temp.extend([x.strip() for x in text.split(',') if not x.strip() == ''])
    samples[i] = list_temp
print(samples)

#output
[['A former employee of the accused company',
  '— — —',
  'offered a statement off the record.'],
 ['He is afraid of two things—spiders and senior prom.Fifty-six bottles of pop on the wall',
  'fifty-six bottles of pop.']]

开发环境

的Python 3.7.0