Home / ubuntu / Questions / 1289324
Accepted
Praveen
Asked: 2020-11-04 05:27:16 +0800 CST

Commandline copy of contents between patterns from a text file to another

  • 772

I have an html file contents copied to a text file in the following form:

... course 10> user_1 </a><br /><a class="reviewlink" href="https://www.foo.com/moodle/mod/quiz/review.php?attempt=1491">Review attempt<...
... course 10> user_2 </a><br /><a class="reviewlink" href="https://www.foo.com/moodle/mod/quiz/review.php?attempt=1498">Review attempt<...

From this using some shell command can I get the contents:

user_1 "https://www.foo.com/moodle/mod/quiz/review.php?attempt=1491"
user_2 "https://www.foo.com/moodle/mod/quiz/review.php?attempt=1498"

to another text file.

Here ... denotes some big unique text.

Edit

More elaborate form for a user:

 <label for="attemptid_1502" class="accesshide">Select attempt</label></td><td class="cell c1 picture" id="mod-quiz-report-responses-report_r1_c1"><a href="https://www.foo.com/moodle/user/view.php?id=380&amp;course=10" class="d-inline-block aabtn"><img src="https://www.foo.com/moodle/pluginfile.php/419/user/icon/moove/f2?rev=6942" class="userpicture" width="35" height="35" alt="Picture of user 1" title="Picture of user 1" /></a></td><td class="cell c2 bold" id="mod-quiz-report-responses-report_r1_c2"><a href="https://www.foo.com/moodle/user/view.php?id=380&amp;course=10">user 1</a><br /><a class="reviewlink" href="https://www.foo.com/moodle/mod/quiz/review.php?attempt=1502">Review attempt</a></td><td class="cell c3" id="mod-quiz-report-responses-report_r1_c3">meethu</td><td class="cell c4" id="mod-quiz-report-responses-report_r1_c4">text</td><td class="cell c5" id="mod-quiz-report-responses-report_r1_c5">Finished</td><td class="cell c6 bold" id="mod-quiz-report-responses-report_r1_c6"><a href="review.php?q=62&amp;attempt=1502">0.00</a></td><td class="cell c7" id="mod-quiz-report-responses-report_r1_c7"><a href="https://www.foo.com/moodle/mod/quiz/reviewquestion.php?attempt=1502&slot=1" id="action_link5fa01d6f2fd9a29" class=""  title="Review response" ><span class="que"><i class="icon fa fa-remove text-danger fa-fw icon"  title="Incorrect" aria-label="Incorrect"></i><span class="notanswered">-</span></span></a></td><td class="cell c8" id="mod-quiz-report-responses-report_r1_c8"><a href="https://www.foo.com/moodle/mod/quiz/reviewquestion.php?attempt=1502&slot=2" id="action_link5fa01d6f2fd9a30" class=""  title="Review response" ><span class="que"><i class="icon fa fa-remove text-danger fa-fw icon"  title="Incorrect" aria-label="Incorrect"></i><span class="notanswered">-</span></span></a></td><td class="cell c9" id="mod-quiz-report-responses-report_r1_c9"><a href="https://www.foo.com/moodle/mod/quiz/reviewquestion.php?attempt=1502&slot=3" id="action_link5fa01d6f2fd9a31" class=""  title="Review response" ><span class="que"><i class="icon fa fa-remove text-danger fa-fw icon"  title="Incorrect" aria-label="Incorrect"></i><span class="notanswered">-</span></span></a></td><td class="cell c10" id="mod-quiz-report-responses-report_r1_c10"><a href="https://www.foo.com/moodle/mod/quiz/reviewquestion.php?attempt=1502&slot=4" id="action_link5fa01d6f2fd9a32" class=""  title="Review response" ><span class="que"><i class="icon fa fa-remove text-danger fa-fw icon"  title="Incorrect" aria-label="Incorrect"></i><span class="notanswered">-</span></span></a></td><td class="cell c11" id="mod-quiz-report-responses-report_r1_c11"><a href="https://www.foo.com/moodle/mod/quiz/reviewquestion.php?attempt=1502&slot=5" id="action_link5fa01d6f2fd9a33" class=""  title="Review response" ><span class="que"><i class="icon fa fa-remove text-danger fa-fw icon"  title="Incorrect" aria-label="Incorrect"></i><span class="notanswered">-</span></span></a></td><td class="cell c12" id="mod-quiz-report-responses-report_r1_c12"><a href="https://www.foo.com/moodle/mod/quiz/reviewquestion.php?attempt=1502&slot=6" id="action_link5fa01d6f2fd9a34" class=""  title="Review response" ><span class="que"><i class="icon fa fa-remove text-danger fa-fw icon"  title="Incorrect" aria-label="Incorrect"></i><span class="notanswered">-</span></span></a></td><td class="cell c13" id="mod-quiz-report-responses-report_r1_c13"><a href="https://www.foo.com/moodle/mod/quiz/reviewquestion.php?attempt=1502&slot=7" id="action_link5fa01d6f2fd9a35" class=""  title="Review response" ><span class="que"><i class="icon fa fa-remove text-danger fa-fw icon"  title="Incorrect" aria-label="Incorrect"></i><span class="notanswered">-</span></span></a></td><td class="cell c14" id="mod-quiz-report-responses-report_r1_c14"><a href="https://www.foo.com/moodle/mod/quiz/reviewquestion.php?attempt=1502&slot=8" id="action_link5fa01d6f2fd9a36" class=""  title="Review response" ><span class="que"><i class="icon fa fa-remove text-danger fa-fw icon"  title="Incorrect" aria-label="Incorrect"></i><span class="notanswered">-</span></span></a></td><td class="cell c15" id="mod-quiz-report-responses-report_r1_c15"><a href="https://www.foo.com/moodle/mod/quiz/reviewquestion.php?attempt=1502&slot=9" id="action_link5fa01d6f2fd9a37" class=""  title="Review response" ><span class="que"><i class="icon fa fa-remove text-danger fa-fw icon"  title="Incorrect" aria-label="Incorrect"></i><span class="notanswered">-</span></span></a></td><td class="cell c16" id="mod-quiz-report-responses-report_r1_c16"><a href="https://www.foo.com/moodle/mod/quiz/reviewquestion.php?attempt=1502&slot=10" id="action_link5fa01d6f2fd9a38" class=""  title="Review response" ><span class="que"><i class="icon fa fa-remove text-danger fa-fw icon"  title="Incorrect" aria-label="Incorrect"></i><span class="notanswered">-</span></span></a></td><td class="cell c17" id="mod-quiz-report-responses-report_r1_c17"><a href="https://www.foo.com/moodle/mod/quiz/reviewquestion.php?attempt=1502&slot=11" id="action_link5fa01d6f2fd9a39" class=""  title="Review response" ><span class="que"><i class="icon fa fa-remove text-danger fa-fw icon"  title="Incorrect" aria-label="Incorrect"></i><span class="notanswered">-</span></span></a></td><td class="cell c18" id="mod-quiz-report-responses-report_r1_c18"><a href="https://www.foo.com/moodle/mod/quiz/reviewquestion.php?attempt=1502&slot=12" id="action_link5fa01d6f2fd9a40" class=""  title="Review response" ><span class="que"><i class="icon fa fa-remove text-danger fa-fw icon"  title="Incorrect" aria-label="Incorrect"></i><span class="notanswered">-</span></span></a></td><td class="cell c19" id="mod-quiz-report-responses-report_r1_c19"><a href="https://www.foo.com/moodle/mod/quiz/reviewquestion.php?attempt=1502&slot=13" id="action_link5fa01d6f2fd9a41" class=""  title="Review response" ><span class="que"><i class="icon fa fa-remove text-danger fa-fw icon"  title="Incorrect" aria-label="Incorrect"></i><span class="notanswered">-</span></span></a></td></tr><tr class="" id="mod-quiz-report-responses-report_r2"><td class="cell c0" id="mod-quiz-report-responses-report_r2_c0"><input id="attemptid_1505" name="attemptid[]" type="checkbox"  value="1505"
   data-action="toggle"
   data-toggle="slave"
   data-togglegroup="quiz-attempts"
   
/>
scripts command-line bash sed copy

2 Answers

  1. Best Answer

    Use python with beautifulsoup4, install it using pip.

    sudo apt install python-pip
pip install bs4

    Put this in a file script.py:

    #!/usr/bin/env python

from bs4 import BeautifulSoup

# Open your file and read+parse its contents with BeautifulSoup
with open('file.html', 'r') as f:

    contents = f.read()
    soup = BeautifulSoup(contents, 'lxml')
  
    # Find and loop all <td class="c2"> elements
    for td in soup.findAll('td', {'class': 'c2'}):
        
        # Find the first <a> element below <td> and get its text value
        user_name = td.find("a", recursive=False).text
        
        # Find the <a class="reviewlink"> element and get its `href`
        review_href = td.find("a", {'class' : 'reviewlink'}, recursive=False).get('href')

        # Print the information as you wish
        print('{} "{}"'.format(user_name, review_href))

    Then run python script.py

    Output:

    user 1 "https://www.foo.com/moodle/mod/quiz/review.php?attempt=1502"
    • 1

  2. XMLStarlet is a set of tools for manipulating XML documents, which is really useful when querying elements using XPath.

    $ xmlstarlet -q fo --html file.html | \
  xmlstarlet sel -t -m '//td[contains(@class,"c2")]' \
    -v 'a[1]/text()' -o \ \" -v 'a[2]/@href' -o \" -n 2> /dev/null

    If you have a well-formed document, you may skip the first line. To query elements, XMLStarlet uses templates. A template can contain multiple sub-options. In this case, we start by -m matching the td element containing the links. The result will then pass to the next sub-option, where we print the value of the links.

    • 0