Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support display download progress bar #525

Merged
merged 20 commits into from
Jan 11, 2024
Merged
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
88 changes: 63 additions & 25 deletions src/download.rs
Original file line number Diff line number Diff line change
Expand Up @@ -171,7 +171,7 @@ pub fn download(url: &str) -> Result<Vec<u8>> {
match handle.get(url).call() {
Ok(response) => {
let mut data = Vec::new();
write_response_with_progress_bar(response, &mut data)?;
write_response_with_progress_bar(response, &mut data, String::new())?;
return Ok(data);
}
Err(ureq::Error::Status(404, r)) => {
Expand Down Expand Up @@ -203,7 +203,11 @@ pub fn download_file(url: &str, path: &PathBuf) -> Result<()> {
for _ in 1..RETRY_ATTEMPTS {
match handle.get(url).call() {
Ok(response) => {
if let Err(e) = write_response_with_progress_bar(response, &mut file) {
if let Err(e) = write_response_with_progress_bar(
response,
&mut file,
path.display().to_string(),
) {
fs::remove_file(path)?;
return Err(e);
}
Expand All @@ -216,8 +220,6 @@ pub fn download_file(url: &str, path: &PathBuf) -> Result<()> {
let retry = retry.unwrap_or(RETRY_DELAY_SECS);
info!("Retrying..");
thread::sleep(Duration::from_secs(retry));
// clean file content every retry
file = OpenOptions::new().write(true).truncate(true).open(path)?;
}
Err(e) => {
fs::remove_file(path)?;
Expand Down Expand Up @@ -286,9 +288,10 @@ pub fn unpack_bins(dir: &Path, dst_dir: &Path) -> Result<Vec<PathBuf>> {
}

/// write Ok(Response) to provided writer with progress bar displaying writing status
pub fn write_response_with_progress_bar<W: Write>(
fn write_response_with_progress_bar<W: Write>(
response: Response,
writer: &mut W,
target: String,
) -> Result<()> {
let total_size = response
.header("Content-Length")
Expand All @@ -299,7 +302,7 @@ pub fn write_response_with_progress_bar<W: Write>(
let progress_bar = ProgressBar::new(total_size);
progress_bar.set_style(
ProgressStyle::default_bar()
.template("[{elapsed_precise}] [{bar:40.cyan/blue}] {bytes}/{total_bytes} {bytes_per_sec} ({eta}) - {msg:.cyan}")
.template("[{elapsed_precise}] [{bar:40.cyan/blue}] {bytes}/{total_bytes} ({eta}) - {msg:.green}")
.unwrap()
.progress_chars("##-"),
);
Expand All @@ -311,35 +314,30 @@ pub fn write_response_with_progress_bar<W: Write>(
break;
}
if let Err(e) = writer.write_all(&buffer[..bytes_read]) {
debug!(
"[{}] [{}] {}/{} {}/s ({}) - {}",
FormattedDuration(progress_bar.elapsed()),
"#".repeat(
(progress_bar.position() * 40 / progress_bar.length().unwrap()) as usize
),
HumanBytes(progress_bar.position()),
HumanBytes(progress_bar.length().unwrap_or(progress_bar.position())),
HumanBytes(progress_bar.per_sec() as u64),
HumanDuration(progress_bar.eta()),
progress_bar.message(),
);
error!("Something went wrong writing data: {}", e)
log_progress_bar(&progress_bar);
if target.is_empty() {
bail!("Something went wrong writing data: {}", e)
}
bail!("Something went wrong writing data to {}: {}", target, e)
};
downloaded_size += bytes_read as u64;
progress_bar.set_position(downloaded_size);
}
progress_bar.finish_with_message("Download complete");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this shows that the download completed even if there was an error. Could you add a unit test where it fails?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have tested in local, when download encount error, won't show "download completed"

log_progress_bar(&progress_bar);
Ok(())
}

fn log_progress_bar(progress_bar: &ProgressBar) {
debug!(
"[{}] [{}] {}/{} {}/s ({}) - {}",
"[{}] [{}] {}/{} ({}) - {}",
FormattedDuration(progress_bar.elapsed()),
"#".repeat(40),
"#".repeat((progress_bar.position() * 40 / progress_bar.length().unwrap()) as usize),
HumanBytes(progress_bar.position()),
HumanBytes(progress_bar.length().unwrap_or(progress_bar.position())),
HumanBytes(progress_bar.per_sec() as u64),
HumanDuration(progress_bar.eta()),
progress_bar.message(),
);
Ok(())
}

/// Read the version (as a plain String) used by the `fuels` dependency, if it exists.
Expand Down Expand Up @@ -402,8 +400,31 @@ pub fn fetch_fuels_version(cfg: &DownloadCfg) -> Result<String> {
mod tests {
use super::*;
use dirs::home_dir;
use std::io::{self, Result};
use tempfile;

struct MockWriter;

impl Write for MockWriter {
fn write_all(&mut self, _: &[u8]) -> Result<()> {
Err(io::Error::new(
io::ErrorKind::Interrupted,
"Mock Interrupted Error",
))
}

fn write(&mut self, _: &[u8]) -> Result<usize> {
Err(io::Error::new(
io::ErrorKind::Interrupted,
"Mock Interrupted Error",
))
}

fn flush(&mut self) -> Result<()> {
Ok(())
}
}

pub(crate) fn with_toolchain_dir<F>(f: F) -> Result<()>
where
F: FnOnce(tempfile::TempDir) -> Result<()>,
Expand Down Expand Up @@ -469,7 +490,7 @@ fuels = { version = "0.1", features = ["some-feature"] }
}

#[test]
fn test_write_response_with_progress_bar() -> Result<()> {
fn test_write_response_with_progress_bar() -> anyhow::Result<()> {
let mut data = Vec::new();
let len = 100;
let body = "A".repeat(len);
Expand All @@ -480,9 +501,26 @@ fuels = { version = "0.1", features = ["some-feature"] }
body,
);
let res = s.parse::<Response>().unwrap();
assert!(write_response_with_progress_bar(res, &mut data).is_ok());
assert!(write_response_with_progress_bar(res, &mut data, String::new()).is_ok());
let written_res = String::from_utf8(data)?;
assert!(written_res.trim().eq(&body));
Ok(())
}

#[test]
#[should_panic]
fn test_write_response_with_progress_bar_throw_error() {
let mut mock_writer = MockWriter;
let len = 9000;
let body = "A".repeat(len);
let s = format!(
"HTTP/1.1 200 OK\r\n\
Content-Length: {}\r\n
\r\n
{}",
len, body,
);
let res = s.parse::<Response>().unwrap();
assert!(write_response_with_progress_bar(res, &mut mock_writer, String::new()).is_ok());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't believe that we need to assert or is_ok() here, no?
If the call fails then the panic will be triggered and the test will fail as expected, or am I misunderstanding?

Copy link
Contributor Author

@Halimao Halimao Dec 20, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

anyhow::bail! doesn't triigger panic
image

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or we can remove [should_panic], just use assert
image

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. If we are able to catch the exact error and assert that the specific error matches what we expect in the test then we should do that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. If we are able to catch the exact error and assert that the specific error matches what we expect in the test then we should do that.

Done

}
}
Loading